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Abstract 


This  research  evaluated  the  effects  of  perturbations  in  time-phased  force 
deployment  data  (TPFDD)  on  airlift  system  performance.  Four  characteristics  of  two 
TPFDD  files  were  perturbed  according  to  a  2*'^  fractional  factorial  experimental  design 
and  then  fed  into  the  Airlift  Flow  Module  (AFM)  simulation.  Specifically,  small, 
constrained  perturbations  were  made  to  the  specified  locations,  the  timeline,  the  amounts 
of  cargo  and  passengers,  and  the  proportion  of  outsize  and  oversize  cargo  categories.  The 
effects  these  perturbations  had  on  AFM  output  were  interpreted  using  factor  analysis.  The 
computed  factor  scores,  when  plotted  against  one  another,  provided  sensitivity  plots, 
graphically  depicting  the  sensitivity  of  the  airlift  performance  response  to  the 
perturbations.  Counter  to  recent  airlift  analysis  discussions  which  promote  completely 
unconstrained  random  TPFDD  generation,  this  research  indicated  that  small,  constrained 
variations  could  cause  potentially  significant  and  unpredictable  changes  in  airlift 
performance.  Of  potential  interest  to  the  simulation  community,  another  by-product  of 
this  research  was  the  potential  to  use  factor  analysis  as  a  verification  and  validation  tool 
for  large,  complex  simulation  models. 
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AIRLIFT  SYSTEM  SENSITIVITY  TO  PERTURBED 
TIME-PHASED  FORCE  DEPLOYMENT  DATA 


I.  Introduction 


General  Issue 

Air  Mobility  Command  (AMC)  makes  resource  and  policy  decisions  for  an  airlift 
system  that  is  driven  by  Time-Phased  Force  Deployment  Data  (TPFDD)  specifying  airlift 
movement  requirements.  A  properly  sequenced  TPFDD  is  the  result  of  a  long  process 
that  takes  12  to  18  months,  and  lists,  in  much  detail,  how  US  forces  will  be  deployed 
(Lund  and  others,  1993:24).  TPFDDs  are  important  to  many  planning  and  decision 
processes  within  the  Department  of  Defense. 

AMC’s  decision  process  typically  requires  insight  fi-om  some  of  the  analytic  tools 
used  by  AMC’s  Studies  and  Analysis  Flight  (AMCSAF).  In  particular,  the  output  from 
their  Mobility  Analysis  Support  System’s  (MASS)  Airlift  Flow  Module  (AFM),  a  large- 
scale  simulation  model,  is  typically  used  to  predict  airlift  system  performance.  AFM 
requires  a  TPFDD  as  input  because,  like  the  airlift  system  itself,  AFM  is  highly  dependent 
upon  specific  transportation  requirements.  Currently,  for  lack  of  a  better  method,  analysis 
of  airlift  system  performance  in  any  given  conflict  is  based  on  the  prioritized  cargo  and 
passenger  movement  requirements  in  only  one  TPFDD. 
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In  considering  the  quality  of  analysis  of  airlift  system  performance  based  on  one 
TPFDD,  several  questions  arise:  Can  AMC  make  the  best  decisions  based  on  a  single 
TPFDD?  How  sensitive  is  the  performance  of  the  airlift  system  to  changes  or  errors  in  the 
TPFDD?  If  we  vary  the  TPFDD,  can  we  gain  a  better  understanding  of  what  is  the 
expected  performance?  Can  an  approach  be  devised  to  reduce  the  uncertainty  in  our 
expectations  of  how  the  airlift  system  will  behave  by  varying  or  perturbing  a  TPFDD?  Is 
there  a  way  to  assess  or  evaluate  airlift  system  performance  across  multiple  scenarios? 

A  recent  example,  though  not  an  AMC  decision,  highlights  some  of  these 
questions.  A  significant  input  to  the  decision  to  purchase  more  C-17  aircraft  was  a  large 
and  detailed  simulation  study,  the  Strategic  Airlift  Force  Mix  Analysis  (SAFMA),  which 
was  based  on  a  single,  much  debated  TPFDD  (AMMP,  1996:  1-26).  Though  the  validity 
of  the  results  of  that  study  are  not  in  question,  one  has  to  ask  if  the  resulting  decision  will 
provide  the  best  airlift  fleet  for  all  possible  future  airlift  taskings?  Is  it  possible  to  place  a 
statistical  envelope  around  the  study  results,  efiectively  bounding  airlift  system 
performance  across  several  similar  scenarios?  Would  the  decision  change  given  a  different 
scenario?  Would  it  prove  to  be  robust  across  multiple  scenarios? 

In  a  briefing  given  to  the  Defense  Acquisition  Board  in  November  of  1995,  the 
major  selling  point  for  the  C-17  was  its  relative  insensitivity  to  a  15  percent  loss  in 
available  MOG,  or  parking  space  (Merrill,  1995).  If  the  TPFDD  were  slightly  different, 
would  the  results  from  this  MOG  resource  reduction  prove  to  be  similar?  Is  it  possible  to 
perturb  the  given  TPFDD,  without  changing  network  resources,  to  obtain  similar  results 
or  to  validate  the  assumed  insensitivity  of  the  results  to  different  scenarios? 
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Translating  the  previous  example  to  military  force  structure  planning  in  general,  it 
appears  that  the  C-17  purchase  decision  was  no  different  than  that  of  buying  the  next 
fighter  or  tank,  or  deciding  the  size  and  composition  of  the  2005  light  infantry  battalion. 
Current  military  force  structure  analysis  requires  the  assumption  of  a  single  scenario  and 
strategy.  This  assumption  comes  in  the  form  of  time-phased  force  deployment  data 
(TPFDD),  which  contains  the  desired  flow  of  cargo  and  personnel  corresponding  to  the 
war  plan  for  that  scenario.  A  typical  TPFDD  is  large  and  detailed,  but  it  has  many 
weaknesses  when  used  as  a  transportation  requirement  database  for  analysis  (Yost,  1995: 
30-33).  In  the  case  of  the  SAFMA  study,  the  only  TPFDD  used  was  the  result  of  the 
Mobility  Requirements  Study  Bottom-Up  Review  Update  (MRS-BURU)  which  was 
completed  in  December  1994  (AMMP,  1995:  1-25). 

Ten  years  ago,  it  was  sufficient  to  plan  and  buy  for  the  one  big  threat,  the  Union  of 
Soviet  Socialist  Republics  (USSR).  At  the  time,  the  assumption  was:  if  we  could  defeat 
the  USSR,  any  other  war  would  be  easier  (Hagin:  1995).  Today’s  military  planning 
cannot  rely  on  the  same  assumption  because  it  is  no  longer  valid.  With  less  money,  a  more 
complex  global  situation,  and  less  of  our  military  stationed  outside  the  Continental  United 
States  (CONUS),  threat-driven  analyses  are  harder  to  accomplish  and  harder  still  to  use  as 
a  basis  for  justifying  substantial  allocations  of  resources  that  take  a  decade  to  acquire  and 
then  will  be  in  service  for  25  years  or  more  (Gebman  et  al,  1994:  35). 

Ensuring  the  robustness  of  decisions  will  require  analysis  across  a  spectrum  of 
possible  scenarios  or  TPFDDs  (Yost,  1995:  33).  Since  TPFDDs  define  a  current  “best” 
strategy  for  a  given  conflict,  they  contain  pertinent  scenario-specific  data.  However,  due 


1-3 


to  the  inherent  uncertainty  in  a  given  TPFDD,  airlift  analysis  should  also  consider 
variations  of  the  given  TPFDD. 

In  this  light,  this  research  effort  considered  randomly  perturbing  two  existing 
TPFDDs  to  provide  different  TPFDD  variations  for  analysis.  The  two  unclassified 
TPFDDs  used  in  this  study  were  provided  by  AMCSAF  in  the  format  required  for  AFM 
input.  This  format  obscures  some  of  the  detail  normally  provided  by  an  actual  TPFDD 
and  contains  only  the  airlift  portion  of  a  complete  TPFDD.  The  AFM  input  file  containing 
the  TPFDD  is  known  as  a  file  63  (see  Appendix  A).  The  argument  up  to  this  point  for  a 
TPFDD  can  be  directly  applied  to  a  file  63  without  loss  of  generality.  From  this  point 
forward,  however,  TPFDD  will  be  used  to  represent  the  original  requirements  listing  in  its 
original  format.  File  63  will  be  used  to  represent  the  pre-processed  airlift  portion  of  the 
TPFDD  that  is  formatted  for  AFM  input. 

Problem  Statement 

Current  analysis  of  the  airlift  system  is  restricted  by  a  reliance  on  the  details  of  a 
given  TPFDD.  AMCSAF  desires  a  methodology  to  understand  and  evaluate  airlift  system 
performance  over  a  variety  of  scenarios,  without  such  a  strong  reliance  on  the  exact 
details  of  a  given  TPFDD. 

Research  Objectives 

The  focus  of  this  research  effort  will  be  on  perturbing  the  two  provided  files  63  to 
examine  the  effects  the  perturbations  have  on  selected  measures  of  effectiveness  (MOE’s). 
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The  desired  approach  follows  these  general  steps,  which  are  discussed  further  in 
Chapter  HI; 

1)  Identify  AFM  output  variables  reflecting  desired  MOE’s 

2)  Analyze  sensitivities  to  AFM  random  seed  using  factor  analysis 

3)  Define  perturbation  schemes 

4)  Analyze  sensitivities  to  perturbations  using  factor  analysis 

This  thesis  documents,  in  detail,  the  process  of  perturbing  a  given  file63.  In 
particular,  it  describes  the  means  by  which  these  perturbations  are  constrained  to  represent 
operationally  realistic  and  transportationally  feasible  TPFDDs.  This  thesis  also  documents 
the  impact  of  this  perturbation  process  on  the  variability  of  the  AFM  output.  The 
application  of  perturbed  TPFDD’s  to  a  robust  analytical  methodology  is  also 
demonstrated.  Finally,  a  discussion  of  results  and  conclusions,  including  the  benefits  of 
this  process  to  future  AMC  airlift  policies  and  force  planning,  is  presented. 

Thesis  Organization 

The  next  chapter  identifies  related  research  efforts  and  provides  background 
information  on  major  topics  pertinent  to  this  research;  the  AFM  simulation,  the  TPFDD 
creation  process,  TPFDD  parameters  and  imcertainty,  airlift  system  performance,  and 
factor  analysis.  Chapter  m  will  discuss  the  details  of  the  outlined  process  and  the 
methodology  used  to  apply  that  process  to  analysis  of  airlift  system  performance.  Chapter 
IV  will  discuss  the  input  and  output  data.  Finally,  Chapter  V  will  present  the  resulting 
recommendations  and  conclusions. 
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n.  Background  Information 


Introduction 

This  chapter  discusses  the  literature  review  and  other  background  information 
pertinent  to:  the  AFM  simulation,  the  TPFDD  creation  process,  TPFDD  parameters  and 
uncertainty,  airlift  system  performance,  and  factor  analysis  techmques. 

Literature  Review 

This  thesis  topic  originated  from  conversations  at  AMCSAF  on  theoretical 
possibilities  of  unconstrained  random  TPFDD  generation  for  airlift  system  performance 
analysis  (Revetta,  1995).  This  thesis  topic  evolved  as  an  exploratory  look  at  the 
possibilities  of  using  randomly  generated  TPFDDs  for  airlift  analysis. 

Though  perturbations  to  the  TPFDD  seem  to  interest  many  in  the  strategic 
transportation  community,  little  has  been  documented  as  to  how  these  perturbations  might 
affect  strategic  lift  capability.  Conversations  with  OSD  PA&E  indicated  they  would  soon 
be  sponsoring  research  into  the  effects  perturbed  TPFDDs  have  on  the  performance  of  all 
transportation  modes  (Schaefer,  1995). 

The  earliest  reference  found,  supporting  the  problem  with  using  only  one  scenario 
for  airlift  analysis,  was  a  dissertation  by  Colgan,  who  conducted  a  cost  effectiveness  study. 
The  relationship  of  cost  effectiveness  to  performance  seems  fairly  direct,  and  the 
arguments  and  observations  Colgan  used  regarding  the  “right  mission”  for  analysis  apply 
equally  well  to  analysis  of  airlift  performance.  He  raised  the  question  of  the  validity  of  a 
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fixed  TPFDD,  and  proposed  using  a  combination  of  TPFDDs  in  evaluating  system 
performance  (Colgan,  1967;  85). 

A  study  by  Ulusoy  and  Uzsoy  indicated  the  lack  of  research  literature  related  to 
robust  strategic  airlift  performance  across  multiple  scenarios  (Ulusoy  and  Uzsoy,  1992: 
275).  They  presented  a  linear  programming  formulation  of  a  strategic  airlift  problem  that 
includes  generating  a  set  of  scenarios  over  a  set  of  possible  demand  patterns.  This  is  the 
only  literature  found  that  addressed  the  problem  of  airlift  system  performance  analysis 
using  a  generated  set  of  scenarios.  Their  model,  however,  solves  an  aircraft  location 
problem  that  is  dependent  upon  total  throughput  as  the  basis  for  the  result  (Ulusoy  and 
Uzsoy,  1992:  266-276).  Besides  simplistic  airlift  system  constraints,  the  difficulty  of 
extrapolating  their  model  to  include  time-phased  data  and  detailed  dimensions  of  airlift 
performance  did  not  seem  appropriate  for  this  research. 

A  practical  and  sometimes  humorous  assessment  of  the  current  state  of  airlift 
system  modeling  and  analysis,  as  expressed  in  a  recent  PHALANX  article,  highhghted  the 
fallacies  involved  with  using  only  one  TPFDD  for  analysis.  The  article  went  further  to 
propose  using  multiple  scenarios  (TPFDDs)  in  pursuit  of  more  robust  analysis  (Yost, 
1995:  30-33). 

Robinson  was  another  proponent  of  using  multiple  scenarios  for  airlift  system 
analysis.  His  contextual  analysis  of  air  transportation  systems  proposed  and  supported 
analysis  of  varying  scenarios,  but  he  completely  disregarded  the  TPFDD  as  the  basis  for 
his  scenarios  (Robinson,  1993). 
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Hanson  successfully  applied  factor  analysis  during  his  analysis  of  the  output  from 
the  MINOTAUR  mobility  simulation  model.  His  research  did  use  a  TPFDD  to  define  the 
scenario,  but  he  did  not  attempt  to  perturb  the  TPFDD  and  his  analytical  objectives  were 
quite  different  than  the  objectives  of  this  thesis  (Hanson,  1990). 

The  AFM  Simulation 

The  Mobility  Analysis  Support  System  (MASS),  maintained  by  AMCSAF,  is  a 
collection  of  computer  models  and  utilities,  written  mostly  in  FORTRAN,  that  provide  for 
detailed  analysis  of  airlift  system  performance.  The  centerpiece  of  MASS  is  the  Airlift 
Flow  Module  (AFM),  a  requirements  driven  simulation  that  plans  missions 

deterministically  and  executes  stochastically  (AFM  Documentation,  1996).  The 

TCQuiTsmcnts  are  contained  in  the  file  63,  which  is  derived  from  the  TPFDD.  These 
requirements  “drive”  the  simulation  processes  because  AFM  deterministically  reads  the 
file  63  sequentially  from  the  top,  line  by  line,  until  it  identifies  the  first  available  and 
compatible  cargo  for  the  next  available  aircraft.  Once  AFM  finishes  planning  the  mission, 
the  aircraft’s  flight  and  ground  times  used  during  execution  of  the  mission  are  drawn  from 
random  distributions.  AFM  release  4  was  used  for  this  research  effort.  The  following 
summary  of  AFM  input,  planning,  execution,  and  output  is  from  an  AMCSAF  point  paper 
provided  to  Congressional  Budget  Office  analysts  in  October  1993  (Merrill,  1993). 

Inputs.  Inputs  to  the  AFM  include; 

-  a  TPFDD  containing  airlift  movement  requirements 

-  an  airlift  network  of  onloads,  offloads,  en-route  stops,  recovery  bases,  and 
home  stations  connected  by  user-defined  routes 

-  an  airlift  fleet  mix  of  different  aircraft  types  identified  by  individual  tail 
numbers 
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-  individual  aircrews  who  must  be  available  to  allow  missions  to  be  floym 

-  logistics  factors  which  account  for  refueling,  maintenance,  and  material 
handling  of  cargo 

-  concepts  of  operation  that  include  strategic  intertheater  airlift,  aerial 
refueling,  intratheater  shuttle  operations,  direct  delivery  operations,  and 
recoveiy/stage  operations 

Planning.  Mission  planning  in  the  AFM  accomplishes: 

-  Prioritization  of  requirements  by  available-to-load  dates  and  required 
delivery  dates 

-  Prioritized  route  selection  and  reservation  for  flight  planning 

-  Marrying  a  specific  aircraft  tail  number  to  the  next  eligible  requirement 

-  Crew  planning  to  ensure  that  only  the  crews  eligible  to  fly  do  fly 

Execution.  Mission  execution  in  the  AFM  simulates: 

-  Typical  sortie  events,  including:  taxi-out,  takeoff,  departure,  en-route 
cruise,  initial  approach,  final  approach,  landing,  taxi-in,  and  ground 
activities  for  every  sortie  of  every  mission 

-  Ground  activity  resource  allocation  and  planned  delays  for:  ramp  space, 
offloading  cargo,  refueling,  maintenance,  unloading  cargo,  and  crew 
changing 

-  Optionally,  detailed  loading  of  each  piece  of  cargo  for  compatibility  with 
doors  and  remaining  space  on  each  aircraft 

-  Crew  activities  and  monitoring  events,  including:  crew  rest,  crew  monthly 
and  quarterly  flying  hour  limits,  crew  availability,  and  searches  for 
imavailable  crews 

Output.  Output  fi’om  the  AFM  includes: 

-  Aircraft-related  statistics,  such  as:  utilization  rate,  payload,  ground  service 
time,  flight  time,  and  system  delays 

-  Aircrew-related  statistics,  such  as:  crew  duty  day,  number  of  crews,  hours 
flown  by  each  crew,  and  crew  availability 

-  Cargo-related  statistics:  total  tons  delivered,  tons  per  day  throughput,  unit 
and  force  closure,  actual  million  ton  miles  per  day  flown,  and  cargo 
remaining  in  backlog 

-  Airlift  network  statistics:  typical  cycle  times,  flying  times,  network  airfield 
use,  MOG  constraints,  and  system  bottlenecks 

r.nnfigiiration  and  Setup.  To  help  minimize  the  setup  time  and  to  ensure  valid 
configurations  for  typical  AFM  inputs,  AMCSAF  provided  all  input  files  for  this  research 
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effort.  The  two  complete  data  sets  provided  were  extracted  from  data  sets  used  in 
previous  AMCSAF  studies.  One  setup  represented  a  large  scale  conflict  in  the  Middle 
East,  similar  to  DESERT  SHIELD/DESERT  STORM.  The  other  setup  represented  a 
much  smaller  scale  conflict  in  the  Caribbean.  These  two  setups  will,  for  simplicity,  be 
called  the  large  and  small  scenarios,  respectively.  Figure  2-1  illustrates  the  differences 
between  the  two  scenarios  in  terms  of  the  total  airlift  requirement. 

Large  Scenario 

Small  Scenario 

0  100000  200000  300000  400000  500000 

□  Total  Passenger  Requirement  (people) 

■  Total  Cargo  Requirement  (tons) 


Figure  2-1.  Large  and  Small  Scenario  Total  TPFDD  Requirements 
Table  2-1  shows  the  difference  in  the  number  of  available  aircraft  in  each  scenario. 
Table  2-1.  Available  Aircraft  (by  type)  for  Research  Scenarios 


1 

C-5A 

C-141B 

C-17 

KC-10 

B-747^ 

B-747P^ 

DC-8 

95 

75 

37n 

60 

90 

40 

Small  Scenario 

28 

56 

15 

28 

0 

11 

0 

Notel.  B-747  is  used  to  r 
Note  2.  B-747P  is  used  to 

epresent  a  Boeing  747  configured  for  cargo  only, 
reoresent  a  Boeinq  747  configured  for  pax  only. 

Airlift  aircraft  types  vary  in  size  and  capability,  and  therefore,  cargo  must 
somehow  be  categorized  to  identify  its  compatibility  with  each  aircraft  t3q)6.  For  airlift, 
typical  categories  are;  outsize  cargo,  oversize  cargo,  bulk  cargo,  and  passengers  (or  pax). 
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Outsize  cargo  is  largest  category  and  requires  use  of  C-5  or  C-17  aircraft.  Oversize  cargo 
exceeds  the  usable  dimensions  of  a  standard  463L  pallet,  but  is  within  the  dimension 
requirements  of  the  C-141B  and  C-130  cargo  doors  and  cargo  compartments.  Oversize 
cargo  requires  a  C-130  or  larger  aircraft,  or  a  freight-configured  wide-body  civil  transport. 
Bulk  cargo  includes  cargo  that  is  within  the  usable  dimensions  of  a  463L  pallet,  and  can  be 

carried  on  most  aircraft  (AMMP,  1996:  1-10). 

Assumptions  about  how  to  use  an  aircraft  that  can  cany  both  passengers  and  cargo 
will  make  a  large  difference  in  the  performance  of  the  airlift  system.  For  this  research, 
from  the  modeling  standpoint,  assumptions  about  cargo  and  passenger  compatibility  are 
coded  in  the  AFM  input  file  32.  Table  2-2  presents  a  these  compatibility  codes. 


Table  2-2.  AFM  Cargo  and  Passenger  Compatibility  with  Modeled  Aircraft  Types 


Cargo  and  Pax  Compatibility  Codes^ 

Outsize 

Oversize 

Bulk 

Pax 

C-5A 

2 

2 

1 

1 

C-141B 

0 

2 

2 

1 

C-17 

2 

2 

2 

1 

KC-10 

0 

1 

2 

0 

B-747^ 

0 

0 

2 

0 

B-747P^ 

0 

0 

0 

2 

DC-8 

0 

0  n 

2 

0 

Note  1 .  B-747  is  used  to  represent  a  Boeing  747  configured  for  cargo  only. 
Note  2.  B-747P  is  used  to  represent  a  Boeing  747  configured  for  pax  only. 
Note  3.  Compatibility  Codes  (See  AFM  File  32  documentation  for  more): 


2  =  cargo  category  is  preferred  by  this  aircraft  type 
1  =  cargo  category  is  compatible  with  this  aircraft  type 
0  =  cargo  category  is  not  compatibie  with  this  aircraft  type. 

Of  all  the  AFM  input  files,  only  the  files  63  (the  input  files  containing  airlift 

requirements)  for  both  the  large  and  small  scenarios  were  changed  during  the  course  of 

the  research.  AMCSAF  suggested  switch  settings  that  limited  the  number  of  output  files 
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and  minimized  runtimes.  Minimal  runtimes  were  beneficial  to  completing  this  research  in 
the  time  allotted.  Another  suggestion  that  was  implemented  was  to  specify  an  early 
termination  date  for  the  simulation.  It  typically  takes  30  to  45  days  for  the  first  sealift 
deliveries  to  arrive  to  a  theater  of  war.  Therefore,  the  first  30  or  45  days  of  the  conflict 
are  known  as  the  airlift  surge  period,  where  airlift  is  responsible  for  delivering  the 
necessary  equipment,  personnel,  and  firepower  to  halt  the  enemy  advance  (AMMP,  1995: 
1-22, 23).  For  this  research,  day  30  was  considered  the  end  of  the  surge  period,  and  both 
the  large  and  small  scenarios  were  terminated  at  that  point  to  gather  the  output  statistics. 

Time-Phased  Force  Deployment  Data  (TPFDD) 

There  are  at  least  two  different  kinds  of  TPFDDs,  basically  classified  by  purpose. 
Those  used  in  the  deliberate  planning  process  for  operational  plan  (OPLAN)  development 
are  very  specific,  receive  a  lot  of  attention,  and  are  firequently  updated  because  they  focus 
against  a  specific  threat  for  a  limited  time  span.  OPLAN  TPFDDs  become  an  annex  to  the 
actual  war  plan  and  contain  the  most  accurate  information  possible  within  each 
requirement  (Hanson,  1990:  5).  Those  used  for  non-OPLAN  purposes,  like  training, 
exercises,  and  long-range  resource  planning  may  not  be  as  specific  within  each 
requirement,  usually  containing  notional  units  and  equipment  that  represent  our  expected 
force  composition  for  some  future  period.  In  future  analysis  studies,  AMCSAF  will  likely 

use  both  kinds  of  TPFDDs  (Steppe,  1995). 

A  simplified  general  discussion  of  the  TPFDD  creation  process  starts  with  the  Joint 

Staff  tasking  the  combatant  command  CINCs  to  develop  an  OPLAN,  then: 

Upon  tasking,  the  CINCs  develop  the  concept  of  operations,  which  is 
passed  along  to  the  theater  service  components  along  with  the  apportioned 
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combat  forces  and  transportation  resources.  The  theater  components 
specify  the  necessary  supporting  forces  and  suppUes,  translating  the  whole 
into  a  time-phased  list  of  cargo  requirements,  or  time-phased  force 
deployment  data  (TPFDD).  (Schanketal,  1991:  13-14) 

Computerized  combat  simulations  are  typically  used  to  assess  the  OPLAN  s  strategy  and 
risk  at  this  point.  If  the  strategy  and  risk  are  satisfactory,  then  the  associated  TPFDD  is 
checked  for  transportation  feasibility  with  at  least  one  computerized  mobility  model 
(Schanketal,  1991:  14).  Transportation  feasibihty  indicates  that  current  transportation 
assets  can  deliver  everything  listed  in  the  TPFDD  in  time  to  effect  a  successful  OPLAN.  If 
not  transportationally  feasible,  the  TPFDD  goes  back  to  the  CINCs,  and  the  process 
becomes  iterative  until  some  certification  of  transportation  feasibility  is  achieved.  Refining 
conferences  are  periodically  held  to  balance  more  detailed  transportation  feasibility  with 
acceptable  risk  on  the  battlefield.  From  start  to  finish,  it  typically  takes  12  to  18  months  to 
create  a  transportationally  feasible  TPFDD  (Lund  et  al,  1993:24). 

Before  AFM  can  use  the  information  contained  in  a  TPFDD,  pre-processing 
utilities  are  used  to  convert  the  TPFDD  to  the  required  input  file,  file  63,  format  (see 
Appendk  A).  The  file  63  format  contains  only  14  fields,  which  is  a  great  reduction  firom 
the  97  fields  in  the  TPFDD  (JOPES  Manual,  1993;  48-55).  Though  not  all  fields  are  used 
in  either  format,  it  is  obvious  that  AFM  does  not  consider  many  of  the  specific  TPFDD 
details. 

The  individual  backgrounds  and  personalities  of  the  various  people  involved  in  the 
creation  of  a  given  TPFDD  have  an  impact  on  the  details  of  that  TPFDD.  Since  different 
generals  will  place  different  priorities  on  the  100,000  or  more  requirement  lines  that  can 


2-8 


comprise  a  large  TPFDD,  it  is  not  unreasonable  to  expect  that  there  could  be  as  many 
TPFDDs  for  a  given  conflict  as  there  are  generals  (Schank  et  al,  1991  ;25).  When  it  comes 
time  to  execute  a  war  plan,  it  is  also  reasonable  to  think  that  the  war  will  not  go  as 
planned.  As  was  seen  in  Desert  Storm,  the  TPFDD  changed  so  frequently  that  the  what, 
when  and  where  aspects  of  the  TPFDD  were  completely  different  from  one  day  to  the 
next  (USGAO,  1993:  24-25).  These  observations  introduce  the  possibility  of  some  degree 
of  uncertainty  within  any  TPFDD.  Uncertainty  in  the  TPFDD,  however,  does  not  imply 
the  TPFDD  is  worthless.  On  the  contrary,  TPFDDs  contain  the  best  information  available 
for  their  intended  purpose,  and  that  is  to  support  the  Joint  Strategic  Planning  System  in 
developing  feasible  operating  plans  (Schank  et  al,  1991:13). 

Airlift  System  Performance 

There  are  many  measures  of  airlift  system  performance  used  depending  upon  the 
question  asked  and  the  data  available.  It  is  important  to  note  that  fighting  unit  combat 
effectiveness  is  the  ultimate  measure  of  airlift  system  performance,  but  there  is  no  direct 
translation  between  individual  pieces  of  cargo  delivered  and  combat  effectiveness 
(Schaefer,  1995).  Combat  unit  closure  statistics  are  the  best  statistics  for  estimating 
combat  effectiveness,  but  are  not  easily  computed.  Since  most  outsize  cargo  is  combat 
firepower,  a  gross  measure  fisr  estimating  combat  effectiveness  is  total  outsize  tons 
delivered. 

Throughput  measures,  aggregate  measures  of  total  tons  or  passengers  delivered, 
seem  to  be  very  common  to  all  recent  airlift  studies.  A  common  throughput  measure  of 
effectiveness  is  the  million-ton-mile  per  day  (MTM/D)  statistic,  though  its  actual 
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usefulness  has  been  much  debated  (Robinson,  1993:10-14;  also  Yost,  1995.  33).  Charts 
depicting  throughput,  in  total  tons  or  numbers  of  passengers  delivered,  over  time  have 
been  interpreted  as  being  preferred  by  Air  Force  leadership  (Robinson,  1993:  12).  The 
Army  troops  on  the  ground  facing  the  enemy,  however,  have  another  measure  of  airlift 
system  performance:  did  they  get  the  right  cargo  at  the  right  place,  ontime? 

Firepower  delivered  a  day  late  is  of  no  use  if  the  war  is  already  over.  The 
timeliness  of  deliveries  is  as  critical  to  combat  effectiveness  as  firepower,  so  several 
measures  of  timeliness  performance  were  included  in  the  data  analyzed  during  this 

research. 

From  AMC’s  perspective,  the  performance  of  the  individual  aircraft  types  is 
critical  information  to  managing  and  improving  the  airlift  system.  Aircraft  utilization  rates, 
average  payloads,  and  cycle  times  are  common  aircraft  performance  measures  of  efficiency 
and  productivity.  Aircraft  utilization  rates  are  typically  broken  down  by  aircraft  type,  and 
reflect  the  average  number  of  hours  each  aircraft  of  that  type  flew  in  an  average  day. 
Unless  many  underlying  assumptions  are  held  constant,  however,  utilization  rates  and 
average  payload  values  can  be  misinterpreted  and  may  not  be  reliable  indicators  of 
performance  (Kowalsky,  1977:  1).  This  observation  formed  the  basis  for  incorporating 
AFM  subroutines  to  compute  cycle  statistics.  One  aircraft  cycle  consists  of  the  events  that 
occur  between  the  time  the  aircraft  leaves  the  blocks  at  one  station  untU  he  returns  to  that 
station  and  is  ready  to  leave  the  blocks  again  (Kowalsky,  1977:  4-1).  Furthermore,  each 
cycle  can  be  broken  down  into  the  flying  cycle  time  and  the  ground  cycle  time.  The  flying 
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cycle  time  is  the  cumulative  number  of  hours  during  the  cycle  in  which  the  aircraft  was 
flying.  The  ground  cycle  time  is  the  remaining  time. 

Airlift  system  performance  is  highly  dependent  upon  crew  performance,  which  is 
another  area  that  is  of  concern  to  AMC  leadership.  Flying  hour  limits,  crew  rest,  crew 
duty  days,  morale,  fatigue,  and  other  crew  limitation  factors  can  have  a  serious  impact  on 
airlift  system  capabilities.  Flying  hour  limits  are  typically  in  the  form  of  maximum  hours 
flown  in  30,  60,  and/or  90  days.  Crew  rest  is  the  number  of  hours,  including  meals,  that 
each  crew  is  entitled  to  between  crew  duty  periods.  Crew  duty  days  limit  the  number  of 
hours  a  crew  can  continuously  perform  flying-related  duties,  and  can  vary  based  on  crew 
composition  or  available  aircraft  equipment,  such  as  the  autopilot.  Unfortunately,  each  of 
these  crew  limitation  factors  has  some  sort  of  waiver  status  that  can  be  exercised  by  the 
various  levels  of  command.  Varying  assumptions  about  these  waivers  in  any  airlift  system 
performance  study  can  have  a  serious  impact  on  the  analyst’s  results  and  conclusions 
(Shirley,  1995). 

There  are  many  models  that  estimate  the  performance  of  the  strategic  airlift 
system.  Most  of  the  current  strategic  mobility  models  (e.g.  ADANS,  JFAST,  CONOP, 
THRUPUT,  and  MIDAS)  simulate  the  airlift  system  in  one  way  or  another,  yet  few  have 
the  flexibility  and  detail  provided  for  by  AFM.  AFM  has  the  primary  advantage  that  it 
simulates  the  airlift  system  to  the  level  of  the  individual  aircraft  and  the  individual 
pieces/pallets  of  cargo.  This  detail,  however,  comes  at  the  cost  of  long  setup  times  and 
long  run  times. 
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Factor  Analysis 


In  analyzing  the  effects  of  TPFDD  perturbations  on  the  selected  MASS  output 
parameters,  factor  analysis  was  used  to  analyze  the  resulting  variance  of  the  output 
parameters.  Factor  loadings  matrices  provided  pertinent  information  on  the  relationships 
between  the  variables  as  they  responded  to  the  TPFDD  perturbations.  In  addition,  visual 
pictures  of  airlift  system  sensitivity  to  the  TPFDD  perturbations  were  found  by  plotting 
the  first  two  factors  against  each  other.  These  “sensitivity  plots”  proved  to  be  useful  tools 
for  sensitivity  analysis. 

The  origin  of  factor  analysis  is  usually  credited  to  Spearman,  who  first  used  the 
term  “factor  analysis”  in  the  context  of  psychological  testing  for  “general  intelligence”  in 
the  early  1900’s  (Basilevsky,  1994:  x).  After  correcting  Spearman’s  model,  Burt 
furthered  the  psychological  applications  of  the  technique  but  only  served  to  discredit  it 
through  fraudulent  publications  concerning  twins  (Basilevsky,  1994:  x-xi).  Statisticians 
continue  to  engender  debate  concerning  the  validity  and  appropriateness  of  factor  analysis 
techniques  (Basilevsky,  1994:  ix).  However,  since  approximately  1970,  there  has  been  an 
increasing  amount  of  literature  on  scientific  and  statistical  applications  of  factor  analysis, 
because  there 

“...are  tasks  to  which  factor  analytic  techniques  are  well  suited.  Besides 
being  able  to  reduce  large  sets  of  data  to  more  manageable  proportions, 
factor  analysis  has  also  evolved  into  a  useful  data-analytic  tool  and  has 
become  an  invaluable  aid  to  other  statistical  models  such  as  cluster  and 
discriminant  analysis,  least  squares  regression,  time/frequency  domain 
stochastic  processes,  discrete  random  variables,  graphical  data  displays, 
and  so  forth  although  this  is  not  always  recognized  in  the  literature 
(Basilevsky,  1994:  ix).” 
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The  source  of  debate  surrounding  factor  analysis  is  attributable  to  a  perceived 
subjectiveness  and  arbitrariness  in  selecting  and  interpreting  a  correlation  matrix  between 
the  factor  scores  and  the  original  variables.  This  correlation  matrix,  called  a  factor 
loadings  matrix  is  one  of  an  infinite  number  of  possible  orientations  or  rotations  of  the 
data.  Standardized  rotation  schemes  have  been  proven  to  optimize  dififerent  aspects  of  the 
factor  loadings  matrix  and  provide  a  mathematical  basis  to  defend  selection  of  one  factor 
loadings  matrix  out  of  the  theoretical  infinite  possibilities  (Basilevsky,  1994:  ix,  xi). 

Factor  analysis  is  basically  a  data  reduction  technique  that  uncovers  common 
dimensions  or  factors  that  link  together  a  set  of  observed  variables  through  analysis  of  the 
variables’  common  variance.  The  formal  factor  analysis  model  describes  each  original 
observable  variable  in  terms  of  a  linear  function  of  unobservable  common  factors  and  a 
single  latent  unique  factor,  as  shown  by  equation  (1)  (Dillon  and  Goldstein,  1984:  55-61). 

X  =  Af+e  (1) 


where 

X  s  p-dimensional  vector  of  observed  responses 
f  s  q-dimensional  vector  of  unobservable  common  factors  (q  «  p) 
A  =  p  X  q  matrix  of  weights  or  factor  loadings 
e  =  p-dimensional  vector  of  imobservable  unique  factors 

Assuming  that  the  common  factors  and  the  unique  factors  are  uncorrelated  with 

themselves  and  each  other,  the  covariance  matrix  of  the  response  vector  X,  denoted  by 

Sxx,  can  be  expressed  as  shown  in  equation  (2)  (Dillon  and  Goldstein,  1984:61-2). 
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Sxx  =  AA^  +  'i' 


(2) 


where 

2xx=  p  X  p  variance-covariance  matrix 
A  =  p  X  q  matrix  of  weights  or  factor  loadings 
AA^  =  common  part  of  variance-covariance  matrix 

^  s  p  X  p  diagonal  matrix  containing  p  unique  variances 

Once  a  solution  to  equation  (2)  is  found,  the  loadings  for  a  given  factor  reflect  the 
nature  of  that  factor  and  is  subject  to  interpretation  by  the  analyst.  This  interpretation  can 
usually  be  enhanced  by  rotating  the  factor  loadings  matrix.  The  rotated  matrix  of  factor 
loadings  represents  an  alternative  interpretation  of  the  data  and  allows  the  analyst  a  degree 
of  flexibility  (Dillon  and  Goldstein,  1984:  88).  As  mentioned  previously,  the  number  of 
such  rotational  transformations,  orthogonal  or  oblique,  is  infinite,  leading  to  criticism  that 
any  solution  chosen  is,  mathematically  speaking,  arbitrary  (Basilevsky,  1994:  xi). 

Selection  of  certain  criterion  functions  can  define  unique  rotations  and  confiront  this 
criticism  with  more  generally  accepted  statistical  practice  (Basilevsky,  1994:  xii). 

A  commonly  used  rotation  criteria  for  the  varimax  rotation,  results  in  a  unique 
rotation  and  allows  for  enhancing  the  interpretability  of  the  loadings  matrix  structure. 

The  varimax  rotation  seeks  to  maximize  the  variation  of  the  squared  factor  loadings  within 
each  factor,  thereby  forcing  the  loading  coeflBcients  to  either  really  high  or  low  values 
(Dillon  and  Goldstein,  1984:  91).  With  not  many  of  the  loadings  falling  into  the  middle 
“grey  area”  it  is  usually  easier  to  find  an  interpretation  fi’om  the  loadings  matrix  structure. 
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in.  Methodology 


Introduction 

This  chapter  highlights  the  three  steps  used  to  evaluate  the  sensitivity  of  airlift 
system  performance  to  perturbing  the  TPFDD: 

1)  Identify  AFM  output  variables  reflecting  desired  MOE’s 

2)  Analyze  sensitivities  to  AFM  random  seed  using  factor  analysis 

3)  Define  perturbation  schemes 

4)  Analyze  sensitivities  to  perturbations  using  factor  analysis 

These  four  steps  form  the  basis  for  a  general  methodology  that  uses  perturbed  TPFDDs  in 

airlift  system  analysis. 

Identify  AFM  Output  Variables  Reflecting  Desired  MOE’s 

In  general,  the  nature  of  the  problem  will  determine  the  measures  of  effectiveness 
(MOE’s)  most  usefiil  to  the  analyst.  For  this  research,  in  the  interest  of  furthering  the 
concept  of  perturbing  the  TPFDD  to  gain  more  useful  insights  into  the  strategic  mobility 
problem,  many  experts  involved  with  strategic  mobility  analysis  were  solicited  to  identify 
which  variables  measuring  airlift  system  performance  should  be  extracted  fi'om  the  AFM. 
Each  expert  had  a  slightly  different  view  of  which  MOE’s  were  most  important.  In  the 
end,  due  to  the  varied  nature  of  the  responses,  it  seemed  that  the  more  general  variables 
fi-om  a  few  key  performance  areas,  would  please  most  everybody.  Four  major  areas  of 
airlift  system  performance  were  considered:  timeliness,  throughput,  aircraft,  and  aircrew. 
Future  research  can  build  on  the  various  areas  of  airlift  performance,  examining  more  of 
the  specific  details. 
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The  AFM  allows  for  user  selection  of  numerous  output  reports  and  variables.  The 
following  AFM  output  variables  were  the  desired  airlift  system  performance  measures  of 
effectiveness  for  this  study: 

Aircraft  Performance  Measures  for  Each  Aircraft  Type 

-  use  rate 

-  flying  time  per  cycle  (or  f-cycle  time) 

-  ground  time  per  cycle  (or  g-qrcle  time) 

-  average  payload 

-  average  million-ton-miles  per  aircraft  per  day  delivered 

Throughput  Measures 

-  outsize  tons  delivered 

-  total  cargo  tons  delivered 

-  total  passengers  delivered 

-  average  million-ton-miles  per  day  delivered  (MTMPD) 

Timeliness  Measures 

-  percent  of  shipments  delivered  ontime 

-  percent  of  shipments  delivered  late 

-  percent  of  shipments  still  open 

-  average  number  of  days  late 

These  logical  airlift  performance  categories  are  important  to  notice  because  the  data 
collected  for  this  research  clustered  into  these  groupings  during  the  AFM  random  seed 
evaluation  phase.  Crew  performance  measures  were  considered,  but  then  discarded  based 
on  suggestions  by  AMCSAF,  due  to  the  longer  run  times  and  the  complexities  of 
understanding  the  different  ways  AFM  handles  crews  (Shirley,  1995).  For  each  run,  the 
AFM  simulation  was  stopped  at  day  30,  the  end  of  the  surge  period  in  simulation  time,  to 
collect  the  selected  performance  measures. 

It  is  obvious  to  see,  from  the  previous  list,  that  the  number  of  variables  measuring 
airlift  system  performance  will  increase  with  more  aircraft  types  in  the  scenario.  For  each 
run  of  the  large  scenario,  which  contains  7  aircraft  types,  43  performance  variables  were 
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recorded.  For  each  run  of  the  small  scenario,  which  contains  5  aircraft  types,  33 
performance  variables  were  recorded. 

Analyze  Sensitivities  to  AFM  Random  Seed  Using  Factor  Analysis 

One  of  the  observed  weaknesses  of  the  AFM  is  the  lack  of  documented 

performance  characteristics.  In  particular,  the  effects  of  its  own  internal  random  seed  on 

the  internal  coded  processes  and  the  output  were  unknown  (Hagin,  1995).  The  AFM  has 

100  well-tested  pseudo-random  streams  from  which  all  stochastic  processes  get  their 

random  values,  as  required.  One  stream  needs  to  be  initialized  as  the  reference  stream  for 

AFM  when  the  simulation  is  started.  All  internal  AFM  processes  that  require  a  random 

value  make  a  call  to  any  one  of  the  100  pseudo-random  streams,  depending  on  the 

displacement  that  process  adds  to  the  reference  stream  value.  Therefore,  changing  the 

reference  stream  will  affect  all  of  AFM’s  internal  stochastic  processes.  There  was  no 

documentation  as  to  the  expected  variance  in  any  output  variable  due  to  running  AFM 

with  different  initial  stream  values  (Hagin,  1995).  Before  applying  random  perturbations 

to  input  data,  understanding  the  sensitivities  of  the  model  to  its  internal  random  seed  was 

essential.  The  variance  of  all  the  output  variables  from  50  runs  was  computed  individually 

to  quantify  the  sensitivity  of  each  variable  to  the  random  seed  (see  Appendices  B  and  C). 

When  evaluating  the  perturbations,  the  variance  from  the  random  seed  runs  served 

as  a  useful  reference  measure  for  internal  random  noise,  a  characteristic  of  any  stochastic 

model.  According  to  Law  and  Kelton: 

“Each  run  of  a  stochastic  simulation  produces  only  estimates  of  a  model’s 
true  characteristics  for  a  particular  set  of  input  parameters.  Thus  several 
independent  runs  of  the  model  will  probably  be  required  for  each  set  of 
input  parameters  to  be  studied  (Law  and  Kelton,  1991 :  115).” 
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Since  each  perturbation  of  the  TPFDD  was  a  new  “set  of  input  parameters  to  be  studied,” 
being  able  to  distinguish  between  variance  caused  by  the  model,  or  characteristics  of  the 
model,  and  variance  caused  by  the  TPFDD  perturbations  was  critical  to  understanding  the 
effects  of  those  perturbations. 

For  each  of  the  two  scenarios  provided,  the  number  of  runs  must  be  at  least  as 
large  as  the  number  of  recorded  variables  to  ensure  enough  degrees  of  freedom  to  estimate 
the  necessary  values  for  factor  analysis  (Bauer,  1995),  Considering  this  fact,  50  runs  with 
different  initial  random  streams  were  made  for  both  the  large  and  the  small  scenarios. 

Data  from  the  large  and  small  scenarios  were  analyzed  separately  in  an  attempt  to 
identify  what  caused  the  observed  changes  in  the  output.  Some  variables  were  eliminated 
by  observation,  before  using  factor  analysis.  Some  of  the  cycle  time  variables  were 
eliminated  by  observation  because  they  didn’t  output  consistently  from  all  50  runs.  For 
some  runs,  AFM  did  not  obtain  enough  cycles  for  each  aircraft  type  to  be  able  to  calculate 
a  value.  Other  variables  were  consistently  output  as  zero  values  because  they  referred  to 
something  that  could  not  happen  in  the  model.  For  instance,  B-747  passenger  aircraft 
could  not  carry  cargo  in  the  simulation,  so  average  payload  and  MTMD  for  this  aircraft 
type  were  always  zero.  Other  variables  were  eliminated,  iteratively,  by  comparing  their 
performance  against  known  uniform  random  noise  variables  during  factor  analysis.  A 
reliable  set  of  performance  variables  was  identified  by  computing  the  eigenvalues  of  the 
correlation  matrix  and  observing  that  all  eigenvalues  were  greater  than  zero  (Bauer, 

1995).  Once  a  reliable  set  of  performance  variables  was  determined,  the  dimensionality  of 
the  performance  measures  were  further  reduced  by  solving  for  the  underlying  common 
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factors  with  factor  analysis.  The  varimax  rotation  was  used,  without  exception,  to 
enhance  the  interpretability  of  the  factor  loadings  matrix. 

Since  only  the  random  seed  changed  from  run  to  run,  the  simulations  internal 
stochastic  processes  should  be  the  source  of  variation  in  all  output  variables.  In 
comparison,  the  main  assumption  of  factor  analysis  is  that  there  is  an  unknown  set  of 
factors  causing  all  variables  in  your  data  set  to  change.  Therefore,  the  unknown  factors, 
computed  from  output  resulting  from  multiple  runs  where  the  only  difference  between 
runs  is  the  change  in  the  random  seed,  should  reflect  the  internal  stochastic  processes 
because  they  are  the  only  physical  explanation  for  the  observed  changes  in  the  data  set.  If 
the  factor  loading  matrix  allows  for  interpretation  of  the  underlying  factors  in  terms  of  the 
known  stochastic  processes,  the  process  of  verification  can  be  accomplished.  To  a  lesser 
degree,  there  also  seems  to  be  some  extension  of  this  concept  to  the  process  of  validation. 
The  following  definitions  of  verification  and  validation  are  provided  to  support  their 
intended  meaning  to  this  discussion  (Pritsker,  1986:  11): 

Verification:  The  process  of  establishing  that  the  computer  program  executes  as 

intended. 

Vahdation:  The  process  of  establishing  that  a  desired  accuracy  or 

correspondence  exists  between  the  simulation  model  and  the  real 
system. 

As  will  be  shown  in  Chapter  IV,  interpretations  of  the  factor  loadings  matrices  accurately 
reflected  the  coded  processes  of  the  simulation.  In  turn,  these  processes  were  able  to  be 
explained  or  related  to  real-world  processes  that  normally  occur  in  an  airlift  system.  For  a 
simulation  model  that  has  had  no  formal  validation  effort,  these  interpretations  provided  a 
type  of  verification  and  validation  for  AFM. 
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Since  most  of  the  selected  performance  variables  were  averages  or  functions  of 
averages,  the  distributions  of  these  variables  were  expected  to  be  fairly  normal  due  the 
central  limit  theorem  (Mendenhall  et  al,  1990;  319).  Therefore,  multivariate  normality 
was  generally  expected  from  this  set  of  performance  variables.  Since  the  distributions  of 
the  variables  should  have  been  preserved  during  the  transformation  to  the  smaller  set  of 
factors,  the  distributions  of  the  factors  were  also  expected  to  resemble  normal 
distributions.  Therefore,  plotting  two  factors  on  orthogonal  axes  should  have  presented  a 
nice  “shotgun  pattern”  representing  a  bivariate  normal  distribution.  In  this  case,  any 
pattern  other  than  a  single  shotgim  pattern  indicated  an  unexpected  violation  of 
multivariate  normality  and  possibly  a  quirk  in  the  model. 

Define  Perturbation  Schemes 

This  section  describes  the  perturbations  made  to  the  various  parameters  of  the 
file  63  and  explains  the  experimental  design  used.  Ideally,  the  goal  was  to  perturb  the 
file  63  enough  so  that  possible  TPFDD  variations  were  accounted  for,  but  not  so  much 
that  the  perturbations  became  operationally  unrealistic.  For  each  line  in  the  file  63  the 
following  variables  were  perturbed  according  to  an  experimental  design:  aerial  port  of 
embarkation  (APOE),  aerial  port  of  debarkation  (APOD),  available-to-load  date  (ALD), 
required  delivery  date  (RDD),  outsize  tons,  oversize  tons,  bulk  tons,  and  numbers  of 
passengers.  Appendix  D  contains  the  FORTRAN  code  used  to  generate  the  perturbed 
files  63  for  this  research. 

APOE  and  APOD  Perturbation.  The  onload  locations  specified  in  a  TPFDD  can 
be  considered  uncertain  because  peacetime  exercises  and  deployments  move  US  forces 


3-6 


around  the  globe  on  a  regular  basis.  In  addition,  recent  military  downsizing  and 
consolidation  of  US  forces  to  an  ever  smaller  number  of  military  bases  further  confounds 
the  accuracy  of  onload  locations  in  any  TPFDD  created  before  these  recent  changes  were 
made.  Offload  locations  can  also  change  because  the  TPFDD  never  executes  as  planned. 
As  was  seen  in  Desert  Shield/Storm,  frequent  changes  to  the  TPFDD  caused  confusion  as 
to  where  certain  cargo  and  personnel  were  supposed  to  go  (USGAO,  1993:  24-25). 

Since  all  APOE/APOD  pairs  in  the  file  63  must  have  defined  route  segments  and 
other  supporting  information  in  several  other  AFM  input  files,  all  APOE/APOD 
combinations  used  in  the  original  file  63  were  stored  in  a  list,  along  with  the  great  circle 
distance  between  the  two  points.  For  each  line  of  the  file  63,  the  APOE/APOD  pair  was 
swapped  with  a  randomly  selected  pair  from  the  saved  list.  Only  those  APOE/APOD  pairs 
with  a  great  circle  distance  within  ten  percent  of  the  original  distance  were  eligible  for 
random  selection.  This  perturbation  changed  the  distribution  of  cargo  and  passengers 
around  the  network,  but  should  have  left  the  average  ton-distance  workload  requirement 
about  the  same,  providing  a  basis  for  comparison.  This  perturbation  tests  the  model’s 
sensitivity  to  varying  distributions  of  cargo  and  passengers  between  the  limited  network 
resources. 

AT.D  and  RDD  Perturbation.  Historically,  the  priorities  of  the  requirements  and 
the  actual  cargo  delivery  schedule  change  frequently  and  end  up  being  drastically  different 
from  the  initial  TPFDD  (Lund  et  al,  1993:  24-25).  In  addition,  the  time  schedule  of 
onloads  and  offloads  has  been  compromised  since  the  TPFDD  was  first  checked  for  gross 
transportation  feasibility,  because  the  primary  way  to  make  a  TPFDD  transportationally 


3-7 


feasible  is  to  stretch  out  the  timeline.  The  relative  order  of  the  majority  of  TPFDD 
requirements  are  probably  pretty  close  to  the  original  concept  of  operations,  but  the 
timeline  most  likely  is  not. 

When  scheduling  missions  in  simulation  time,  AFM  starts  at  the  top  of  the  file  63 
and  finds  the  first  load  that  is  compatible  with  the  aircraft  type  that  has  an  available-to- 
load  date  (ALD)  equal  to  or  less  than  the  current  simulation  time.  It  is  important  to  notice 
that  AFM  does  not  consider  the  required  delivery  date  (RDD)  during  the  simulation’s 
execution.  ALD  is  only  considered  to  see  if  a  line  of  cargo  is  eligible  to  be  airlifted. 
Compared  to  all  requirements  with  an  ALD  less  than  or  equal  to  the  current  simulation 
time,  the  higher  a  requirement  is  in  the  file  63,  the  higher  its  priority.  Therefore,  the 
file  63  will  be  sorted  by  RDD  to  keep  the  earliest  required  delivery  dates  at  the  top  of  the 
list.  The  ALD/RDD  perturbation  scheme  adds  plus  or  minus  one  day  to  the  ALD  for  each 
requirement,  then  adjusts  the  RDD  to  keep  the  number  of  allowable  delivery  days  the 
same. 

Because  the  AFM  output  variables  selected  were  expected  to  be  very  sensitive  to 
this  perturbation,  the  maximum  magnitude  of  the  perturbation  was  kept  fairly  small 
(Shirley,  1995).  For  each  line  of  the  file  63,  a  randomly  selected  integer  value  of -1,  0, 
or  1  was  added  to  the  available-to-load  date  (ALD)  and  to  the  required  delivery  date 
(RDD).  Negative  values  for  ALD  or  RDD  were  reset  to  zero  due  to  model  restrictions. 
The  perturbed  lines  were  re-sorted  by  RDD,  then  by  ALD,  and  then  by  a  sequence  number 
that  reflected  where  the  line  was  in  the  original  sequence.  This  method  of  sorting  kept  all 
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the  lines  with  the  same  new  ALD  and  RDD  in  the  same  prioritized  sequence  as  in  the 
original  file  63. 

Load  Strain  Perturbation.  The  size  of  the  task,  in  terms  of  tons  of  cargo  and 
number  of  passengers,  seems  somewhat  uncertain  (Robinson,  1993  ;  7).  Plausible  changes 
in  the  nature  offerees  required  for  any  given  scenario  (i.e.  replacing  light  infantry  with  a 
mechanized  brigade  in  some  scenarios,  or  sending  in  Air  Force  interdiction  units  first 
instead  of  Marine  units)  can  change  total  cargo  and  passenger  requirements  (Yost,  1995: 
33).  In  such  situations,  the  possibility  of  increasing  the  required  amounts  of  cargo  and  the 
required  rate  of  cargo  deliveries  seems  likely.  How  much  more  can  we  strain  the  airlift 
system  and  still  get  reasonable  performance?  Can  we  get  more  performance? 

For  this  perturbation  scheme,  the  tonnage  for  each  of  the  cargo  categories  and  the 
number  of  passengers,  for  each  requirement,  were  increased  by  ten  percent.  This  increase 
in  load  requirement  strained  the  airlift  system  by  increasing  the  required  rate  and  demand 
for  deliveries. 

Perturbing  the  Proportion  of  Outsize  vs.  Oversize  Cargo.  In  addition  to  not 
knowing  the  exact  size  of  each  requirement,  knowledge  of  pre-positioned  supplies  can 
change  over  time,  invalidating  some  of  the  assumptions  made  during  the  TPFDD  creation 
process.  Under  current  pre-positioning  assumptions,  the  proportions  for  the  various 
categories  of  airlift  cargo  during  the  airlift  surge  period  are  roughly:  15%  bulk  cargo, 

15%  outsize  cargo,  and  70%  oversize  cargo  (Schaefer,  1995).  Figure  3-1  illustrates  the 
actual  proportions  of  the  cargo  categories  for  the  two  scenarios  included  in  this  research. 
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Small  Scenario  Large  Scenario 


Figure  3-1.  Surge  Period  Cargo  Proportions  for  Research  Scenarios. 

Both  scenarios  show  a  larger  proportion  of  bulk  cargo  and  a  lower  proportion  of 
oversize  cargo  than  the  expected  rough  proportions  mentioned  above,  probably  due 
arbitrary  selection  of  the  30-day  mark  as  the  end  of  the  surge  period  in  this  research. 
Current  thoughts  on  changes  to  our  national  pre-positioning  strategy  almost  double  the 
amount  of  outsize  cargo  and  subtract  an  equivalent  tonnage  from  the  oversize  category 
(Schaefer,  1995).  Therefore,  for  this  perturbation  scheme,  the  amount  of  outsize  cargo  in 
each  line  of  the  file  63  was  doubled.  An  equivalent  tonnage  was  subtracted  from  the 
oversize  cargo  value,  and  was  carried  over  to  subsequent  lines,  if  necessary,  to  keep  the 
total  outsize  plus  oversize  tonnage  value  constant. 

Experimental  Design.  For  the  experimental  design,  each  of  these  four 
perturbations  was  considered  a  deterministic  effect.  An  orthogonal  2^^  fractional  factorial 
design  was  selected  to  allow  for  imbiased  estimation  of  the  effects  of  the  four  perturbation 
schemes,  while  also  estimating  the  effects  from  the  possible  interaction  of  any  two  of  these 
perturbation  schemes  (Box  and  Draper,  1987:  148-166).  The  design  encompassed  8 
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runs,  where  the  number  of  replications  for  each  run  was  an  input  variable  to  the  generation 
program  (see  Appendix  D).  Ten  replications  were  actually  conducted.  During  testing  of 
the  individual  perturbation  code  segments,  additional  files  63  were  created  to  validate  the 
code.  These  files  63  were  saved  and  added  to  the  design,  as  design  points  8  and  9, 
because  they  existed  and  they  allowed  for  direct  measurement  of  the  two  random  effects. 
The  design  matrix  is  shown  in  Table  3-1,  where  0  represents  the  effect  being  off,  and  1 
represents  the  effect  being  on.  For  example,  all  runs  corresponding  to  design  point  1  were 
made  using  a  file  63  that  was  perturbed  by  both  the  load  strain  perturbation  and  the 
outsize/oversize  proportion  perturbation. 


Table  3-1 .  Experimental  Design  for  Evaluating  TPFDD  Perturbation  Effects 


Design 

Point 

TPFDD  Perturbation  Effects  I 

APOE/APOD 

ALD/RDD 

Load  Strain 

Out/Over  Proportion 

0 

0 

0 

0 

0 

1 

0 

0 

1 

1 

2 

0 

1 

0 

1 

3 

0 

1 

1 

0 

4 

1 

0 

0 

1 

5 

1 

0 

1 

0 

6 

1 

1 

0 

0 

7 

1 

1 

1 

1 

8* 

0 

1 

0 

0 

9* 

1 

0 

0 

0 

*Not  part  of  the  original  24-1  factorial  design,  added  after  the  fact 


Analyze  Sensitivities  to  Perturbations  Using  Factor  Analysis 

Ten  replications  of  each  of  the  ten  design  points  resulted  in  output  fi'om  100  AFM 
runs.  Factor  analysis  techniques  were  used  to  reduce  the  dimensionality  of  the  selected 
MOE’s.  For  both  the  large  and  small  scenarios,  each  of  the  resulting  factors  showed  fairly 
straightforward  interpretability  and  defined  key  areas  of  performance.  The  random  seed 


3-11 


runs  provided  an  interpretation  reference  for  the  factor  loadings  matrix  from  the 
perturbation  runs.  The  interpretation  of  most  factors  repeated  the  logical  groupings  of  the 
performance  variables,  as  outlined  at  the  beginning  of  this  chapter.  Since  factors  represent 
orthogonal  and  independent  dimensions,  the  separation  of  the  individual  performance 
categories  on  separate  factors  indicated  total  airlift  system  performance  was  a  nearly  linear 
function  of  the  independent  contributions  from  each  performance  category. 

Factor  plots,  labelled  as  “sensitivity  plots”  in  this  thesis,  graphically  showed  the 
relative  effects  corresponding  to  each  of  the  perturbations.  Ten  of  the  50  random  seed 
rrms  were  included  in  this  portion  and  served  very  well  as  a  reference  measure  of  the 
model’s  internal  random  noise,  clearly  identifying  which  perturbations  were  significant. 
Explanation  of  the  relative  sensitivities  of  both  large  and  small  scenarios  provided  further 
understanding  of  the  internal  processes  of  the  AFM  model. 
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IV.  Data  Description  and  Analysis 


Introduction 

This  chapter  presents  the  data  developed  and  used  in  this  thesis.  A  summary  of  the 
related  analysis  conducted  on  that  data  is  also  provided.  Data  and  analysis  of  50  runs, 
where  the  starting  random  seed  value  changed  from  run  to  run,  are  presented  for  both 
scenarios.  The  application  of  factor  analysis  to  these  random  seed  runs  indicated  potential 
for  using  factor  analysis  during  verification  and  validation  processes  for  large  stochastic 
simulation  models.  After  the  section  on  the  random  seed  run  data,  the  data  and  analysis  of 
100  perturbation  runs  are  presented.  The  application  of  factor  analysis  to  these 
perturbation  runs  provided  a  unique  sensitivity  analysis  tool.  The  resulting  sensitivity  plots 
graphically  showed  the  impact  of  each  of  the  experimental  design  points  relative  to  the 
baseline  performance  measured  in  the  random  seed  runs.  The  summary  at  the  end  of  each 
section  identifies  the  common  links  found  between  the  two  scenarios. 

File  63  Data 

This  section  provides  a  summary  of  the  two  scenarios  used  during  this  research. 
Information  on  the  small  scenario  is  presented  first,  followed  by  the  large  scenario. 

Small  Srenarin  The  small  scenario  file  63  contains  1568  requirement  lines  which 
specify  75,854  tons  of  cargo  and  139,480  passengers  (pax)  destined  primarily  for  islands 
in  the  Caribbean.  Table  4-1  presents  a  summary  of  the  file  63  by  listing  the  amount  of 
cargo  and  number  of  pax  for  each  aerial  port  of  debarkation  (APOD).  The  ICAO 
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their  respective  airfields.  MOG,  in  the  last  column  of  Table  4-1,  stands  for  maximum-on- 
the-ground,  and  represents  a  ground  resource  constraint  where  a  MOG  value  of  1  is 
equivalent  to  the  parking  space  required  to  park  and  service  one  C-141B  sized  aircraft 
within  a  specified  time  window.  MOG  is  commonly  used  as  a  planning  tool  to  reflect  how 
many  aircraft  can  simultaneously  be  on  the  ground  at  one  airfield,  and  is  often  confused 
with  parking  spots  available.  However,  the  relationship  between  MOG  and  parking  spots 
is  not  one-to-one  because  aircraft  larger  than  a  C-141B  may  require  more  than  one  MOG 
resource. 


Table  4-1 .  Cargo  and  Passenger  Destinations  in  Small  Scenario 


Aerial  Ports  of  Debarkation 

ICAO 

Cargo 

Pax 

MOG 

Luis  Munoz  Marin  Inti,  San  Juan 

TJSJ 

75847 

7 

Norman  Manley  Inti/Kingston,  Jamaica 

MKJP 

17353 

17527 

5 

Montego  Bay/Sangster,  Jamaica 

MKJS 

7208 

13680 

4 

Port  au  Prince,  Haiti 

MTPP 

6840 

11540 

2 

Great  Inagua/Matthew  Town,  Bahamas 

MYIG 

6223 

6008 

1 

George  Town/Owen  Roberts  Inti,  Cayman  Is 

MWCR 

2859 

1001 

2 

Roosevelt  Roads  NS,  Puerto  Rico 

TJNR 

1504 

2150 

8 

Macdill  AFB,  Florida 

KMCF 

771 

991 

2 

Key  West  NAS,  Florida 

KNOX 

625 

179 

2 

Homestead  AFB,  Florida 

KHST 

534 

838 

2 

Huriburt  Field,  Florida 

KHRT 

295 

193 

2 

McGuire  AFB,  New  Jersey 

KWRI 

285 

170 

80 

Dyess  AFB,  Texas 

KDYS 

231 

249 

20 

Tyndall  AFB,  Florida 

KPAM 

145 

103 

2 

Eglin  AFB,  Florida 

KVPS 

48 

118 

2 

Orlando  Inti,  Florida 

KMCO 

44 

139 

2 

Moody  AFB,  Georgia 

KVAD 

15 

0 

8 

Miami  Inti,  Florida 

KMIA 

4 

51 

2 

Patrick  AFB,  Florida 

KCOF 

0 

16 

2 

Warner  Robins  AFB,  GA 

KWRB 

0 

10 

42 

Charleston  AFB/Muni,  South  Carolina 

KCHS 

0 

8607 

54 

Jacksonville  NAS,  Florida 

KNIP 

0 

58 

4 

Seymour  Johnson  AFB,  Florida 

KGSB 

0 

5 

8 

ITOTALS 

75854 

139480 

263 
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Notice  that  Table  4-1  is  sorted  in  descending  order  of  the  total  cargo  tons  value. 
The  difficulty  in  dehvering  the  cargo  and  passengers  for  this  scenario  can  be  identified  by 
focusing  on  the  first  seven  airfields.  These  first  seven  airfields  receive  96%  of  the  cargo 
and  92%  of  the  passengers  into  a  cumulative  MOG  value  of  29,  which  is  only  1 1%  of  the 
total  MOG  available  at  destinations  in  this  scenario. 

The  small  MOG  values  for  the  majority  of  the  destination  airfields  in  this  scenario 
did  prove  to  limit  the  overall  airlift  performance.  For  instance,  the  only  ^craft  in  this 
scenario  that  prefers  to  carry  passengers  is  the  Boeing  747,  which  required  a  MOG  of  two 
in  order  to  use  an  airfield.  The  C-5,  C-141B,  and  C-17  aircraft  carried  passengers,  but  not 
nearly  as  efficiently;  and  only  the  C-141B  or  the  C-17  could  deliver  passengers  into  an 
airfield  with  a  MOG  of  one.  Consequently,  the  arrival  rate  of  passengers  at  Great  Inagua 
(MYIG),  with  a  MOG  of  one,  was  fairly  low. 

Large  Scenario.  The  large  scenario  file  63  contains  43 1  requirement  lines  which 
specify  402,796  tons  of  cargo  and  498,369  passengers  (pax)  destined  primarily  for  the 
\fiddle  East.  Table  4-2  presents  a  summary  of  the  file  63  by  listing  the  amount  of  cargo 
and  number  of  pax  for  each  aerial  port  of  debarkation  (APOD).  Notice  that  Table  4-2  is 
sorted  in  descending  order  of  the  total  cargo  tons  value.  Compared  to  the  small  scenario, 
the  distribution  of  the  cargo  and  passengers  between  the  APODs  is  a  little  better.  In  the 
large  scenario,  for  instance,  96%  of  the  cargo  and  91%  of  the  passengers  are  received  into 
the  first  sk  airfields  in  Table  4-2;  but  these  six  airfields  have  a  cumulative  MOG  value  of 
50,  which  is  75%  of  the  total  MOG  available.  Though  the  overall  distribution  seems 
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better,  one  exception  is  the  seemingly  disproportionate  values  for  cargo  and  pax  destined 
for  OEJB  with  a  MOG  value  of  1 . 


Table  4-2.  Cargo  and  Passenger  Destinations  in  Large  Scenario. 


Aerial  Ports  of  Debarkation 

ICAO 

Cargo 

Pax 

Dhahran  International 

OEDR 

308228 

306122 

26 

Jubail  Air  Base 

OEJB 

31079 

81384 

1 

King  Khalid  Military  City 

OEKK 

15642 

31421 

2 

Diego  Garcia  NSF 

FJDG 

13560 

3836 

4 

Doha  International 

OTBD 

11548 

21423 

4 

King  Abdul  Azziz  International 

OEJN 

9205 

6276 

13 

Al  Dhafra  Air  Base 

OMAM 

5714 

6474 

4 

Bahrain  International 

OBBI 

5330 

9272 

4 

Seeb  International 

OOMS 

1490 

4031 

4 

Riyadh  Military 

OERY 

1000 

28130 

5 

ITOTALS 

402796 

498369 

67 

Compared  to  the  small  scenario,  the  MOG  values  for  most  of  the  destination 
airfields  in  the  large  scenario  are  greater.  The  total  cargo  ton  and  passenger  requirement 
is  much  larger  than  in  the  small  scenario.  Also,  the  average  distance  fi-om  in  the  U.S.  to 
the  destinations  is  much  greater  in  the  large  scenario  than  the  small  scenario.  Therefore, 
as  opposed  to  the  small  scenario’s  preference  for  the  performance  of  the  smaller  aircraft 
types,  the  large  scenario  was  expected  to  benefit  more  from  the  larger,  longer  range 
aircraft  types. 

Random  Seed  Run  Data 

In  considering  how  to  determine  if  perturbations  to  the  file  63  caused  more 
variance  than  would  be  expected  from  the  AFM  stochastic  processes,  information  on  the 
effects  of  AFM’ s  internal  random  processes  on  AFM  output  was  desired  as  a  baseline  for 
comparison.  Because  there  was  no  documentation  as  to  how  much  variation  can  be 
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expected  from  the  AFM  simulation  when  the  initial  random  seed  is  changed,  the  effects  of 
changing  the  random  seed  on  selected  output  variables  was  investigated  as  a  prelude  to 
the  analysis  of  perturbation  effects.  The  information  in  this  section  and  in  Appendices  B 
and  C  provide  the  first  known  documentation  of  the  effects  of  changing  random  seeds  on 
AFM  output.  For  both  scenarios,  10  of  the  50  runs  conducted  in  this  section  were  used  as 
a  reference  basis  for  interpreting  the  perturbation  effects,  as  shown  in  the  next  section. 

The  analysis  of  the  perturbation  effects  can  stand  on  its  own  and  does  not  explicitly  rely  on 
any  information  from  this  section.  However,  the  information  presented  here  provides 
insight  into  how  the  AFM  random  processes  work,  demonstrating  how  factor  analysis 
might  be  used  for  stochastic  simulation  model  verification  and  validation. 

Small  Srftnarin  Thirty-three  measures  of  effectiveness  were  extracted  from  the 
AFM  output  for  each  of  the  50  runs  conducted.  Of  the  33  selected  variables,  8  were 
discarded  as  unusable.  The  aircraft  cycle  time  statistics  were  output  as  zero  when  there 
weren’t  enough  cycles  completed  by  the  respective  aircraft  types.  Only  the  C-17  and  the 
C-141B  aircraft  seemed  to  complete  enough  cycles  to  generate  non-zero  cycle  statistics 
for  all  50  runs.  Since  the  B-747P  aircraft  were  not  allowed  to  carry  cargo  in  the 
simulation,  the  B-747P  cargo  statistics  were  zero  for  all  50  runs.  The  eight  discarded 
variables  were: 

C-5  A  f-cycle  time  B-747P  f-cycle  time 

C-5  A  g-cycle  time  B-747P  g-cycle  time 

KC- 1 0  f-cycle  time  B-747P  average  payload 

KC- 1 0  g-cycle  time  B-747P  MTM/D  per  aircraft 
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The  remaining  25  variables  plus  four  injected  random  noise  variables  were  analyzed 
iteratively,  using  PROC  FACTOR  in  SAS  with  a  VARIMAX  rotation.  Comparing  the 
factor  loadings  of  the  25  measures  of  effectiveness  to  the  factor  loadings  of  the  4  noise 
variables,  three  more  variables  could  not  be  distinguished  from  random  noise  during  this 


process,  and  were  discarded  along  with  the  four  injected  noise  variables: 

C- 1 4  IB  f-cycle  time  %  Shipments  still  open 

C-17  f-cycle  time 

In  addition,  MJM/D  delivered  and  %  Shipments  late  were  discarded  because  they  were 


more  than  99  percent  correlated  with  Total  cargo  tons  delivered  and  %  Shipments  ontime, 
respectively.  There  was  concern  that  such  a  high  degree  of  correlation  between  variables 
measuring  the  same  type  of  performance  would  tend  to  bias  the  factor  analysis  results. 

The  remaining  20  variables  still  sufficiently  represented  the  areas  of  airlift  performance  to 


provide  useful  information.  The  measures  of  effectiveness  studied  for  this  scenario  were; 


C-5  A  use  rate 

C-5  A  average  payload 

C-5  A  MTM/D  per  aircraft 

C-17  use  rate 
C-17  g-cycle  time 
C-17  average  payload 
C-17  MTM/D  per  aircraft 


C-141B  use  rate 
C-141B  g-cycle  time 
C-141B  average  payload 
C-141B  MTM/D  per  aircraft 

KC-10  use  rate 
KC-10  average  payload 
KC-10  MTM/D  per  aircraft 


B-747P  use  rate  Outsize  tons  dehvered 

Passengers  delivered 

%  Shipments  ontime  Total  cargo  tons  delivered 

Average  days  late 


Due  to  the  large  difference  in  magnitude  between  the  variables,  the  correlation  matrix  was 
used  instead  of  the  variance-covariance  matrix  during  factor  analysis  (see  Equation  2). 
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When  the  variables  under  consideration  are  measured  in  different  units,  scale  effects  can 
influence  the  composition  of  the  derived  factors  (Dillon  and  Goldstein,  1984:  36).  In 
such  cases,  the  data  are  standardized  and  the  correlation  matrix  is  used  in  place  of  the 


variance-covariance  matrix  (Dillon  and  Goldstein,  1984;  36).  Appendix  B  contains  the 
raw  output  data  from  the  50  runs  conducted,  plus  summary  statistics  and  the  computed 
factor  scores.  As  a  summary  of  Appendix  B,  Table  4-3  presents  the  summary  statistics  for 
each  of  the  20  performance  variables.  The  coefficient  of  variation  is  a  statistical  tool 
defined  by  dividing  the  standard  deviation  by  the  mean.  The  coefficients  of  variation  are 
expressed  as  percentages  and  for  18  of  the  variables  were  at  or  below  2.5%.  The  number 
of  each  aircraft  type  is  indicated  in  parentheses. 


Table  4-3.  Summary  Statistics  for  Small  Scenario  Random  Seed  Runs. 


iiMii 

Variable 

Mean 

Standard 

Deviation 

Coefficient 
of  Variation 

C-5A  (28) 

Use  Rate 

2.3800 

0.0571 

2.4% 

Average  Payload 

70.7858 

0.3962 

0.6% 

MTM/D 

0.0182 

0.0004 

2.4% 

C-141B(56) 

8.2620 

0.1383 

1.7% 

11.4538 

0.0861 

0.8% 

27.4190 

0.1606 

0.6% 

MTM/D 

0.0371 

0.0006 

1.5% 

C-17  (15) 

Use  Rate 

10.7980 

0.4068 

3.8% 

14.4398 

0.2791 

1.9% 

53.6086 

0.8635 

1 .6% 

MTM/D 

0.0526 

0.0039 

7.4% 

KC-10(28) 

Use  Rate 

1.4180 

0.1792 

12.6% 

41.6798 

0.7252 

1.7% 

MTM/D 

0.0005 

10.3% 

B-747P  (11) 

Use  Rate 

4.8760 

0.0847 

1.7% 

■HI 

Outsize  Tons 

39.6607 

0.5% 

Pax 

56636.8000 

361 .8051 

0.6% 

72335.8400 

1643.5637 

2.3% 

Timeliness 

iPercent  Ontime 

84.4330 

0.1045 

0.1% 

0.6746 

0.0065 

1.0% 

The  C-17  use  rate  and  MTM/D  variables  had  slightly  higher  coefficients  of 
variation,  probably  due  to  the  fact  that  there  are  so  few  of  them  in  this  scenario,  relative  to 
the  numbers  of  other  cargo  aircraft.  Randomly  selecting  one  C-17  out  of  15  vrill  have  a 
greater  effect  on  its  use  rate  than  randomly  selecting  one  C-5  or  C-141B  out  of  the  28  or 
56  aircraft,  respectively,  in  the  scenario. 

The  KC-10  use  rate  and  MTM/D  variables  have  the  highest  coefficients  of 
variation.  These  two  variables  also  probably  suffer  from  an  effectively  small  number  of 
aircraft.  Though  there  are  28  KC-lO’s  in  the  small  scenario,  they  are  not  introduced  until 
day  13,  equating  to  just  under  17  aircraft  available  per  day  when  averaged  over  the  full 
30-day  period.  In  addition,  the  magnitudes  of  the  mean  values  for  the  KC-10  use  rate  and 
MTM/D  were  relatively  small  values,  where  any  measurable  change  was  a  significant 
percentage  of  the  mean  value. 

Factor  analysis  reduced  dimensionality  of  the  data  from  20  inter-related  variables 
to  5  independent  factors,  as  shown  in  Table  4-4.  Since  the  factor  analysis  model  attempts 
to  account  for  as  much  of  the  total  variance  in  the  data  as  possible  by  identifying  common 
variance  between  the  variables,  the  amount  of  total  variance  accounted  for  by  the  model  is 
an  indicator  of  the  goodness  of  the  model.  In  this  case,  common  variance  between  the 
variables,  caused  by  the  five  unknown  factors,  accounts  for  71 .9%  of  the  total  variance. 
The  portion  of  common  variance  accounted  for  by  each  of  the  five  factors  is  listed  at  the 
bottom  of  Table  4-4. 
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Table  4-4.  Small  Scenario  Factor  Loadings  Matrix  from  50  Random  Seed  Runs. 


Factor 

1 

2 

C-5A  use  rate 

-0.063 

0.315 

Billcgl 

C-5A  average  payload 

-0.108 

0.085 

0.094 

0.022 

C-5A  MTMs/aircraft/day 

-0.050 

0.230 

-0.184 

0.082 

0.867 

-0.062 

-0.112 

0.102 

0.206 

0.240 

-0.088 

-0.022 

0.025 

C-1416  average  payload 

-0.179 

0.121 

-0.676 

-0.154 

-0.308 

C-141B  MTMs/aircraft/day 

KEES 

0.008 

-0.355 

-0.068 

0.011 

C-17  use  rate 

0.098 

_ 

0.083 

-0.046 

0.157 

C-17  g-cycle  time 

-0.738 

-0.201 

-0.111 

-0.033 

0.071 

C-17  average  payload 

msm 

-0.406 

-0.216 

^222^ 

C-17  MTMs/aircraft/day 

0.088 

0.086 

0.072 

KC-10  use  rate 

0.171 

0.917 

0.206 

0.016 

-0.087 

KC-10  average  payload 

0.018 

-0.605 

0.084 

0.481 

-0.240 

KC-10  MTMs/aircraft/day 

0.183 

0.219 

0.185 

-0.098 

B-747P  use  rate 

0.099 

0.111 

0.224 

-0.562 

-0.292 

Outsize  tons  deiivered 

0.372 

-0.050 

0.139 

-0.245 

Passengers  delivered 

0.104 

0.174 

-0.009 

0.085 

Total  cargo  tons  delivered 

0.173 

0.102 

0.028 

0.089 

%  Shipments  ontime 

-0.062 

-0.053 

-0.034 

-0.720 

0.078 

Average  days  late 

0.099 

0.059 

-0.003 

Eigenvalue 

6.2425 

2.4011 

2.3369 

Common  Variance  (%) 

43.4% 

16.7% 

16.3% 

The  five  factors  described  in  Table  4-4  are  variables,  containing  most  of  the 
information  from  all  20  original  performance  variables.  Each  of  the  50  runs  can  now  be 
represented  by  five  values,  called  factor  scores,  that  are  computed  from  its  20 
corresponding  performance  values.  Factor  scores  for  each  run  are  included  in 
Appendfac  B.  The  significance  of  the  actual  numeric  value  of  a  particular  factor  score  can 
be  best  understood  by  interpreting  the  relationships  between  that  factor  and  the 
performance  variables,  as  shown  in  the  factor  loadings  matrix.  Since  each  loading  value  is 
a  correlation  coefiBcient  between  one  factor  and  one  performance  variable,  interpretation 
stems  from  identifying  which  variables  are  most  highly  correlated  with  the  individual 
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factors.  Loadings  highlighted  in  Table  4-4  with  a  bold  box  were  the  largest  values  for 
their  respective  rows  and  were  considered  significant  loadings. 

By  looking  at  Table  4-4,  one  of  the  interesting  results  was  how  the  factor  loadings 
tended  to  cluster  by  the  type  of  performance  measure  (as  described  in  Chapter  III); 
aircraft,  throughput,  or  timeliness.  These  loading  clusters  conveniently  aided 
interpretation,  relating  most  factors  to  the  performance  of  one  or  two  individual  aircraft 
types.  Only  those  variables  with  loadings  greater  than  0.500  (there  is  one  exception)  were 
used  to  interpret  and  label  each  of  the  factors  as  a  performance  index.  Insight  into  how 
AFM’s  stochastic  processes,  particularly  mission  execution,  affect  AFM  output  can  be 
interpreted  from  the  relationships  between  variables  that  load  on  the  same  factor. 


Factor  1.  C-1 7/C-141B  and  General  Throughput  Index. 

-  Significance.  In  general,  a  high  factor  1  score  reflects;  high  cargo  and 
passenger  throughput,  high  C-17  performance,  and  high  C-141B  performance. 

-  Insight:  Most  of  the  variance  in  cargo  and  passenger  throughput  values  can  be 
explained  by  AFM’s  more  frequent  use  of  C-17  and  C-141B  aircraft  over  other 
aircraft  types.  The  cause  of  this  more  frequent  use  is  the  result  of  randomly 
drawing  shorter  flight  and  ground  times  for  these  aircraft  types  during  execution, 
allowing  them  to  be  scheduled  more  frequently.  Selection  of  aircraft  for  mission 
scheduling  in  AFM  is  first-in  first-out,  depending  on  when  the  aircraft  enter  the 
AFM  mission  scheduler’s  queue.  Aircraft  enter  this  queue  after  completing  a 
mission  and  after  a  random  ground  time.  Runs  that  have  a  high  factor  1  score 
processed  C-17  and  C-141B  aircraft  more  frequently  than  those  runs  with  low 
factor  1  scores.  Opposite  signs  are  typically  called  “contrasts”  in  factor  analysis. 
The  C-17  g-cycle  time  contrast  with  the  positive  performance  variables  makes 
sense  because  the  less  time  a  C-17  sits  on  the  ground,  the  more  cargo  and 
passengers  it  can  move.  An  interesting  peculiarity  that  shows  up  here  is  the 
apparent  positive  correlation  between  C-141B  use  rate  and  C-17  use  rate,  without 
any  indicated  competition  for  MOG  or  cargo.  In  reality,  the  C-17  can  cany 
everything  the  C-141B  can,  but  more  efficiently.  So,  a  higher  C-17  use  rate  should 
typically  mean  a  lower  C-141B  use  rate.  This  peculiarity  is  important  because  it 
led  to  the  unexpected  discovery  of  correlated  bi-modal  distributions  for  these  two 
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variables.  Even  more  unexpectedly,  this  bi-modal  characteristic  applied  to  all  eight 
variables  which  load  highly  upon  this  factor.  Discussion  of  this  “bi-modal 
problem”  is  pursued  later  in  this  section. 

Factor  2.  KC-10  Index. 

-  Significance:  In  general,  a  high  factor  2  score  reflects  high  KC-10  performance. 

-  Insight:  Unrelated  to  throughput  measures  primarily  due  to  the  fact  that 
KC-lO’s  are  not  available  until  day  13  of  the  scenario.  Since  output  statistics  were 
measured  at  day  30,  KC-lO’s  had  a  limited  time  to  contribute  significantly  to  the 
overall  throughput  performance.  Further  limiting  their  contributions  to  throughput 
performance,  over  70%  of  the  bulk  cargo  had  already  been  delivered  before  the 
first  KC-lO’s  were  available,  so  there  wasn’t  much  bulk  cargo  left  for  the  KC-lO’s 
to  cany. 

Factor  3.  C-5A/C-141B  Oversize  and  Bulk  Cargo  Contrast. 

-  Significance:  In  general,  a  high  factor  3  score  reflects  high  C-5  use  and  low 
C-141B  payloads.  Conversely,  a  low  factor  3  score  reflects  greater  C-141B 
payloads  and  less  C-5  use. 

-  Insight:  Since  C-5  and  C-141B  aircraft  shared  the  capability  to  carry  oversize 
and  bulk  cargo,  this  factor  reflects  a  sort  of  competition  for  which  aircraft  type 
carried  this  common  cargo.  Since  there  was  a  finite  amount  of  cargo,  those  runs 
where  C-5  aircraft  were  used  more,  they  carried  more  cargo  and  left  the  C-141B 
aircraft  with  less  to  carry.  In  those  runs  with  a  high  factor  3  score,  C-5  use  rate 
and  MTM/D  were  high,  causing  a  reduction  in  C- 141  payload.  In  runs  with  low 
factor  3  scores,  C-141B  aircraft  were  flying  with  greater  payloads  and  C-5  use 
rates  were  lower. 

Factor  4.  Lateness  Index. 

-  Significance:  A  high  factor  4  score  reflects  more  late  deliveries  and  low  B-747P 
use.  A  low  factor  4  score  reflects  more  timely  deliveries  and  high  B-747P  use. 

-  Insight:  This  factor  was  interpreted  as  a  lateness  index  instead  of  a  timeliness 
index  because  the  late  variable  was  positive.  In  general,  it  seems  that  variations  in 
lateness  (timeliness)  were  fairly  independent  of  the  variations  in  the  types  of 
aircraft  selected  for  each  line  of  cargo  and  independent  of  the  variations  in  overall 
throughput.  The  exception  obviously  was  the  B-747P  aircraft,  which  were 
configured  to  carry  only  passengers.  B-747P  aircraft  carried  a  large  majority  of 
the  passengers  in  this  scenario.  The  variance  in  the  lateness  seems  to  be  somewhat 
explained  by  B-747P  usage  because  all  the  aircraft  types  were  being  used  more 
efficiently  by  carrying  what  they  were  designed  to  carry.  More  specifically,  runs 
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that  used  the  B-747P  more  led  to  more  timely  deliveries  because  they  freed  up  the 
other  aircraft  for  cargo  only.  Military  cargo  planes  were  not  built  for  pax  and 
don’t  deliver  them  as  efficiently  as  the  B-747P. 

Factor  5.  C-5A  Outsize  Index. 

-  Significance.  In  general,  a  high  factor  5  score  reflects  greater  use  of  C-5  aircraft 
to  carry  outsize  cargo. 

-  Insight:  Since  C-5s  carried  the  majority  of  the  outsize  cargo  in  this  scenario,  it 
makes  sense  that  variations  in  C-5  payloads  affected  the  variations  in  outsize  cargo 
throughput.  For  most  bases  in  this  small  scenario,  C-5  aircraft  were  realistically 
modeled  to  take  1.5  to  2.0  C-141B  equivalent  MOG  resources.  Larger  payloads 
on  the  C-5  meant  longer  ground  times,  in  general,  affecting  the  number  of  C-141B 
aircraft  that  could  obtain  service  or  park  at  the  same  airports.  The  weak 
competition  indicated  (C-141B  ground  cycle  times  increase  with  increasing  C-5 
payloads)  seems  to  reflect  the  competition  between  these  two  aircraft  for  airport 
resources  (parking,  servicing,  etc.).  This  competition  did  not  affect  the  other 
aircraft  types  as  much  because  of  the  relatively  large  numbers  of  C-5  and  C-141B 
aircraft  in  this  scenario. 

Another  productive  method  of  using  factor  analysis  to  understand  the  effects  of 
AFM’s  internal  random  seed  was  plotting  all  50  runs  on  factor  by  factor  plots  (i.e.  a  plot 
of  factor  2  scores  by  factor  1  scores).  Since  the  first  two  fectors  accounted  for  most  of 
the  total  variance,  they  also  provided  the  most  information  about  the  model. 

Most  of  the  20  performance  variables  were  averages  or  functions  of  averages,  and 
the  distribution  of  these  variables  were  expected  to  be  fairly  normal  due  to  the  Central 
Limit  Theorem  (Mendenhall  et  al,  1990:  319).  Though  multivariate  normaUty  is  not  a 
requirement  for  a  data  set  when  using  factor  analysis,  it  was  generally  expected  from  this 
set  of  performance  variables.  Since  the  distributions  of  the  variables  should  have  been 
preserved  during  the  transformation  to  the  smaller  set  of  factors,  the  distributions  of  the 
factors  were  also  expected  to  resemble  normal  distributions.  Therefore,  plotting  two 
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factors  on  orthogonal  axes  should  have  presented  a  nice  “shotgun  pattern”  representing  a 
bivariate  normal  distribution.  In  this  case,  any  pattern  other  than  a  single  shotgun  pattern 
indicated  an  unexpected  violation  of  multivariate  normality. 

Factor  1  scores  were,  in  fact,  distinctly  bi-modal  and  non-normal,  as  can  be  seen  in 
Figure  4-1.  Plots  of  any  of  the  other  factors  against  factor  1  showed  a  similar  breakout  of 
the  data  into  two  distinct  groups.  The  first  question  investigated  was  whether  or  not  this 
bi-modal  split  was  statistically  significant.  A  statistical  T-test,  at  the  0.05  significance 
level,  for  equal  means  between  the  two  groups  of  factor  1  scores  indicated  that  the  split 
was  indeed  statistically  significant.  Furthermore,  similar  T-tests  on  all  8  variables  that 
have  high  loadings  on  factor  1  had  the  same  result.  Indicating  two  statistically  distinct 
groups  within  the  fifty  runs. 


C-17/C-141Band  General  Throughput  Index  Score 

Figure  4-1.  Plot  of  Factor  2  Scores  by  Factor  1  Scores  (Small  Scenario) 
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The  next  question  investigated  was  the  significance  of  the  magnitude  of  the  split. 
Since  the  factor  scores  are  standardized  values  by  their  computation,  a  factor  score  of  1 .0 
would  indicate  the  equivalent  of  one  standard  deviation.  The  means  of  the  two  groups,  in 
terms  of  their  fector  1  score,  are  separated  by  approximately  2.0.  This  equivalent  of  two 
standard  deviations  was  considered  significant.  Looking  back  at  the  coefQdents  of 
variation  for  the  eight  variables  that  load  on  factor  1  (see  Table  4-3),  a  two  standard 
deviation  split  equates  to  less  than  5%  of  the  mean  values.  Though  statistically  significant, 
the  question  of  practical  significance  of  a  5%  split  in  any  of  these  variables  arose. 
AMCSAF  considered  this  split  to  be  practically  insignificant,  but  nonetheless  important  to 
a  better  imderstanding  of  their  model  (Hagin,  1996). 

Further  investigations  into  the  cause  of  this  bi-modal  split  point  to  the  model’s 
mission  scheduling  heuristic  as  a  possible  source.  In  particular,  when  the  mission 
scheduler  attempts  to  match  a  cargo  load  to  an  available  aircraft,  there  is  a  minimum 
amoimt  of  cargo  that  is  considered  too  small,  or  too  trivial,  to  dispatch  an  aircraft;  to  get 
that  cargo.  This  minimum  amount  of  cargo  is  defined  as  the  aircraft’s  trivial  load  limit. 
Cargo  that  is  below  the  trivial  load  limit  for  all  aircraft  basically  gets  ignored  by  the  AFM 
simulation.  There  is  a  trivial  load  collection  heuristic  included  in  the  version  of  AFM  used 
for  this  research  that  attempts  to  correct  for  this  deficiency  in  the  model,  but  apparently,  it 
did  not  work  well  for  3 1  of  the  50  runs  conducted.  The  trivial  load  collection  heuristic 
attempts  to  fill  any  aircraft  that  has  unused  payload  capacity,  only  if  it  has  first  found  a 
non-trivial  load.  A  large  majority  of  requirement  lines  in  the  small  scenario  file  63  are 
trivial  for  the  larger  aircraft.  Even  for  the  C-141B,  which  has  the  smallest  trivial  load  limit 
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of  7  tons,  there  were  over  300  lines  that  would  not  be  picked  up  directly  by  dispatching  a 
C-141B  aircraft.  To  illustrate.  Figure  4-2  shows  a  plot  of  total  cargo  dehvered  over  time 
from  one  run  on  each  side  of  this  bi-modal  split.  The  two  runs  seem  to  be  fairly  equal 
until  about  day  24,  when  one  run  essentially  delivered  the  last  line  of  cargo  and  totaled 
only  71000  tons.  The  other  run  continues  to  deliver  cargo  for  about  3  more  days  because 
it  did  a  better  job  of  collecting  trivial  loads  over  the  first  24  days  and  has  more  cargo  to 
deliver. 


Figure  4-2.  Plot  of  Total  Cargo  Tons  Delivered  Over  Time  for  Two  Selected  Runs 

Considering  that  the  group  of  runs  on  the  high  side  of  the  split  had  more  C-141B 
deliveries  than  the  group  on  the  low  side,  the  mission  scheduler  dispatched  aircraft  to 
more  lines  of  cargo  because  the  C-141B  had  the  smallest  trivial  load  limit.  In  these  cases, 
the  mission  scheduler  also  had  more  opportunity  to  collect  trivial  loads  because  each  time 
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an  aircraft  was  dispatched  to  a  cargo  requirement,  any  unfilled  payload  capacity  on  that 
^craft  was  an  opportunity  for  a  trivial  load  somewhere  nearby.  The  group  of  3 1  runs  on 
the  low  side  of  the  split  had  more  widebody  aircraft  making  deliveries  than  the  group  of 
19  runs  on  the  high  side.  With  more  widebody  aircraft  making  deliveries,  there  were 
fewer  opportunities  for  trivial  cargo  to  be  collected  and  greater  opportunities  for  cargo 
requirements  to  be  considered  trivial. 

Based  on  these  observations,  the  AFM  mission  scheduling  heuristic  seems  to  be 

incompatible  with  the  detailed  structure  of  the  file  63  provided  with  the  small  scenario. 

This  file  63  divides  a  fairly  small  amoimt  of  cargo  into  approximately  1500  requirement 

lines,  where  many  lines  were  small  enough  to  be  delivered  only  as  trivial  loads  with  the 

trivial  load  collection  heuristic,  if  they  were  delivered  at  all  by  the  AFM  simulation. 

Large  Scenario.  Forty  three  measures  of  effectiveness  were  extracted  fi’om  the 

AFM  output  for  each  of  the  50  runs  conducted.  Of  the  43  selected  variables,  2  were 

output  fi’om  AFM  as  zero  for  aU  50  runs  and  were  discarded: 

B-747P  average  payload  B-747P  MTM/D  per  aircraft 

The  remaining  41  variables  plus  four  injected  random  noise  variables  were  analyzed 

iteratively,  using  PROC  FACTOR  in  SAS  with  a  VARIMAX  rotation.  Comparing  the 

factor  loadings  of  the  41  measures  of  effectiveness  to  the  factor  loadings  of  the  noise 

variables,  8  more  variables  could  not  be  distinguished  fi-om  random  noise  during  this 

process,  and  were  discarded  along  with  the  4  injected  noise  variables: 

C-5  f-cycle  time  C-141B  f-cycle  time 

C- 1 7  f-cycle  time  KC- 1 0  f-cycle  time 

DC-8  f-cycle  time  B-747  f-cycle  time 

B-747P  f-cycle  time  %  Shipments  still  open 
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In  addition,  MTM/D  delivered  and  %  Shipments  late  were  discarded  because  they  were 
more  than  99  percent  correlated  with  Total  cargo  tons  delivered  and  %  Shipments  ontime, 
respectively.  There  was  concern  that  such  a  high  degree  of  correlation  between  variables 
measuring  the  same  type  of  performance  would  tend  to  bias  the  factor  analysis  results. 

The  remaining  31  variables  still  sufficiently  represented  the  areas  of  airlift  performance  to 
provide  useful  information.  The  measures  of  effectiveness  studied  for  this  scenario  were: 


C-5  A  use  rate 
C-5  A  g-cycle  time 
C-5  A  average  payload 
C-5  A  MTM/D  per  aircraft 

C-17  use  rate 
C-17  g-cycle  time 
C-17  average  payload 
C-17  MTM/D  per  aircraft 

B-747  use  rate 
B-747  g-cycle  time 
B-747  average  payload 
B-747  MTM/D  per  aircraft 

B-747P  use  rate 
B-747P  g-cycle  time 

%  Shipments  ontime 
Average  days  late 


C-141B  use  rate 
C-141B  g-cycle  time 
C-141B  average  payload 
C-141B  MTM/D  per  aircraft 

KC-10  use  rate 
KC-10  g-cycle  time 
KC-10  average  payload 
KC-10  MTM/D  per  aircraft 

DC-8  use  rate 
DC-8  g-cycle  time 
DC-8  average  payload 
DC-8  MTM/D  per  aircraft 

Outsize  tons  delivered 
Passengers  delivered 
Total  cargo  tons  delivered 


As  in  the  small  scenario,  due  to  the  large  difference  in  magnitude  between  the  variables, 
the  correlation  matrix  was  used  during  factor  analysis.  Appendbc  C  contains  the  raw 
output  data  from  the  50  runs  conducted,  plus  summary  statistics  and  the  computed  factor 
scores.  Table  4-5  presents  the  summary  statistics  from  Appendbc  C.  In  this  case,  the 
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coefficients  of  variation  for  all  3 1  variables  were  at  or  below  2.9%.  The  number  of  each 
aircraft  t5^e  available  to  this  scenario  is  indicated  in  parentheses. 


Table  4-5.  Summary  Statistics  for  Large  Scenario  Random  Stream  Rims. 


Performance 

Standard 

CoefTicient 

Category 

Variable 

Mean 

Deviation 

of  Variation 

C-5A{95) 

Use  Rate 

10.9300 

0.4% 

42.4572 

0.1918 

0.5% 

60.1660 

0.9441 

1.6% 

MTM/D 

0.1196 

0.0019 

1.6% 

C-141B(75) 

Use  Rate 

12.4040 

0.0832 

0.7% 

33.5337 

0.4602 

1.4% 

20.6706 

0.2348 

1.1% 

MTM/D 

0.0425 

0.0004 

1.0% 

C-17(51) 

Use  Rate 

14.6780 

0.0679 

0.5% 

23.0392 

0.3829 

1.7% 

42.9998 

0.4856 

1.1% 

MTM/D 

0.1126 

0.0013 

1.2% 

Use  Rate 

12.6220 

0.0764 

0.6% 

29.4185 

0.3921 

1.3% 

33.8134 

0.9073 

2.7% 

MTM/D 

0.0898 

0.0025 

2.8% 

B-747  (60) 

Use  Rate 

9.4100 

0.0647 

0.7% 

36.4595 

0.4470 

1.2% 

65.0894 

0.4501 

0.7% 

MTM/D 

0.1245 

0.0015 

1.2% 

B-747P  (90) 

Use  Rate 

9.5060 

0.1202 

1 .3% 

46.1696 

0.9782 

2.1% 

DC-8  (40) 

lUse  Rate 

9.3860 

0.1429 

1.5% 

36.0512 

1.0513 

2.9% 

riVi  fs  H  5^  W  M:  S  B 

17.3942 

0.1517 

0.9% 

MTM/D 

0.0317 

2.6% 

Throughput 

Outsize  Tons 

12790.3200 

266.9268 

2.1% 

Pax 

168470.9000 

3289.5428 

2.0% 

119970.7600 

708.3134 

0.6% 

Timeliness 

Percent  Ontime 

14.9680 

0.1407 

0.9% 

Days  Late 

2.5072 

0.0168 

0.7% 

Factor  analysis  reduced  dimensionality  of  the  data  from  3 1  inter-related  variables 
to  10  independent  factors.  In  this  case,  common  variance  between  the  variables,  caused 
by  the  ten  unknown  factors,  accounts  for  82.5%  of  the  total  variance.  As  in  the  small 
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scenario,  the  patterns  of  relationships  between  variables  shown  in  Table  4-6  contributed  to 
understanding  AFM’s  stochastic  processes  during  execution  of  the  large  scenario. 


Table  4-6.  Large  Scenario  Factor  Loadings  Matrix. 


Factor 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

C-5A  use  rate 

0.50 

0.37 

0.26 

-0.27 

-0.20 

-0.08 

-0.11 

C-5A  g-cycle  time 

0.47| 

0.02 

-0.41 

0.30 

gj22y| 

BttiSl 

C-5A  average  payload 

Miinn 

tltg] 

■nra 

HS 

-0.42 

HQ 

-0.09 

0.05 

0.04 

C-5A  MTMs/aircraft/day 

■9 

HU 

■>I4I 

PTOcI 

KEu 

0.03 

C-141Buse  rate 

WKsm 

HS 

1T>F1 

Qsa 

HQ 

C-1 41 B  g-cycle  time 

KiQ 

miss 

0.03 

0.01 

0.93 

Qsa 

C-1 41 B  average  payload 

mss 

■■Brio 

BQ 

C-141B  MTMs/aircrafl/day 

Kl^ 

0.06 

-0.1 3i 

0.73 

-0.13 

0.13 

0.05 

C-1 7  use  rate 

■0.7S 

-0.02 

-0.11 

0.14 

0.09 

0.31 

0.00 

-0.04 

C-1 7  g-cycle  time 

KIS 

■sa 

0.16 

-0.12 

-0.02 

B!2£J 

B&SQ 

-0.54 

0.29 

■>MI 

0.02 

C-1 7  MTMs/aircraft/day 

■ilH 

mss 

-0.55 

0.29 

Q2SU 

0.28 

-0.09 

-0.04 

KC-10  use  rate 

mtm 

-0.06 

0.00 

-0.09 

0.28 

-0.08 

0.19 

0.01 

0.13 

KC-10  g-cycle  time 

1  -0.83 

0.02 

0.01 

0.09 

-0.16 

0.14 

-0.21 

0.06 

0.04 

KC-10  average  payload 

0.94 

0.07 

0.00 

-0.03 

KC-10  MTMs/aircrafl/day 

0.19 

0.18 

KES 

0.04 

HU 

B-747  use  rate 

0.10 

-0.13 

0.06 

0.10 

-0.11 

-0.02 

0.10 

0.09 

B-747  g-cycle  time 

-0.12 

0.30 

-0.10 

0.00 

0.19 

0.04 

HU 

B-747  average  payload 

rogi 

■i»MI 

0.16 

0.16 

031 

B-747  MTMs/aircraft/day 

■CTItl 

-0.06 

-0.26 

0.23 

B-747P  use  rate 

miihi 

0.95 

-0.05 

0.12 

0.02 

0.11 

B-747P  g-cycle  time 

BdOU 

•0.93 

0.03 

-0.11 

0.12 

-0.09 

0.07 

DC-8  use  rate 

mim 

KiS 

»«I*W 

Tg>l 

0.00 

0.00 

-0.03 

0.07 

-0.01 

DC-8  g-cycle  time 

gtWtl 

MM 

mis 

-0.85 

0.04 

-0.04 

-0.01 

0.00 

0.22 

DC-8  average  payload 

row 

Ha 

0.29 

mm 

0.07 

DC-8  MTMs/aircraft/day 

mm 

mEs 

mss 

Ha 

■iWl 

0.10 

-0.19 

0.11 

0.31 

0.21 

Outsize  tons  delivered 

■PTiTil 

MiltW 

lESa 

Ha 

■i»ll»l 

-0.10 

0.20 

0.13 

0.04 

-0.08 

»niB 

HU 

Ha 

HI 

mg 

0.13 

0.14 

miE] 

[Ha 

MiTiW 

0.06 

0.21 

0.04 

0.18 

-0.03 

-0.08 

mssu 

-0.07 

Eigenvalues 

3.97 

3.07 

2.87 

2.75 

2.73 

2.59 

2.39 

2.28 

1.52 

1.42 

Common  Variance  (%) 

15.5% 

12.0% 

11.2% 

10.7% 

10.7% 

10.1% 

9.3% 

8.9% 

5.9% 

5.5% 

Factor  1.  C-5  and  Cargo  Throughput  Index. 

-  Significance:  In  general,  a  high  factor  1  score  reflects  high  cargo  throughput 
and  high  C-5  payload/MTM  performance. 
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-  Insight.  Most  of  the  variance  in  cargo  throughput  values  can  be  explained  by 
AFM’s  more  frequent  scheduling  of  C-5  aircraft  over  other  aircraft  types.  The 
contrast  between  C-5  payloads  and  C-141B  payloads  is  understandable  since  both 
aircraft  compete  for  similar  cargo.  Accounting  for  the  largest  portion  of  common 
variance,  this  factor  reflects  this  scenario’s  preference  for  a  large,  long-range  cargo 
aircraft. 

Factor  2.  C-1 7  andKC-10  Contrast. 

-  Significance.  In  general,  high  factor  2  scores  reflect  high  KC-10  use  and  high 
C-17  ground  cycle  times.  Conversely,  low  factor  2  scores  reflect  high  C-17  use 
and  high  KC-10  ground  cycle  times. 

-  Insight:  This  factor  reflects  competition  between  these  two  aircraft  types  for 
MOG  resomces  and  bulk  cargo.  There  was  also  the  possibility  for  competition 
between  these  two  aircraft  for  oversize  cargo,  but  since  oversize  cargo  was  not 
preferred  by  the  KC-10,  in  the  model,  this  possibility  seemed  remote. 

Factor  3.  B- 747P  and  Passenger  Throughput  Index. 

-  Significance:  In  general,  high  factor  3  scores  reflect  high  passenger  throughput 
performance  and  high  B-747P  use. 

-  Insight:  B-747  aircraft  were  designed  to  carry  passengers,  and  they  do  it  well. 
Almost  all  of  the  variation  in  passenger  throughput  in  this  scenario  is  accounted  for 
by  variations  in  B-747P  performance.  As  in  the  first  factor,  the  large  scenario’s 
preference  for  large,  long-rang  aircraft  is  also  indicated  here. 

Factor  4.  KC-10  Throughput  Index. 

-  Significance:  In  general,  a  high  factor  4  score  reflects  high  KC-10  throughput 
performance. 

-  Insight:  Increased  KC-10  payloads  were  directly  related  to  increased  KC-10 
MTMD,  which  is  not  always  the  case,  as  evidenced  by  Factor  2  in  Table  4-5  from 
the  small  scenario  analysis.  There  was  also  an  apparent  contrast  with  C-17 
throughput  performance,  indicating  a  competition  for  cargo. 

Factors.  DC-8  Index. 

-  Significance:  In  general,  a  high  factor  5  score  reflects  high  DC-8  performance. 

-  Insight:  Increased  use  of  DC-8  aircraft  increased  DC-8  MTM/D  performance 
overall.  The  contrast  with  ground  cycle  times  is  understandable. 
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Factor  6.  C-1 7  and  C-141B  Throughput  Index. 

-  Significance:  High  factor  6  scores  reflects  high  C-17  and  C-141B  throughput. 

-  Insight:  Similar  to  the  small  scenario,  the  C-17  and  C-141B  seemed  to  work  in 
concert  instead  of  in  competition.  Since  C-17  and  C-141B  aircraft  carry  the  same 
cargo,  it  would  have  seemed  more  reasonable  if  there  were  some  indication  of 
competition  for  cargo  or  resources.  This  could  be  the  result  of  a  MOG  tradeoff 
with  the  larger  aircraft  because  each  C-5  and  B-747  require  twice  the  MOG  of  the 
C-17  or  C-1 4  IB.  This  could  also  be  a  repetition  of  similar  phenomena  as  found  in 
the  small  scenario,  presenting  even  more  evidence  of  a  possible  quirk  in  the  model 
that  favors  the  aircraft  with  the  smaller  trivial  load  limits. 

Factor?.  B-747 Index. 

-  Significance:  In  general,  high  factor  7  scores  reflect  high  B-747  performance. 

-  Insight:  Increased  B-747  use  rates  were  positively  correlated  with  increased 
MTMAD,  and  negatively  correlated  with  B-747  ground  cycle  times. 

Factor  8.  C-141B  Ground  Cycle  and  Use  Rate  Contrast. 

-  Significance:  In  general,  high  factor  8  scores  reflect  low  C-141B  use  and  high 
C-141B  ground  cycle  times. 

-  Insight:  This  factor  inversely  reflects  the  amount  of  flying  accomplished  by  the 
C-141B  fleet  in  this  scenario. 

Factor  9.  DC-8  Payload  and  Timeliness  Index. 

-  Significance:  In  general,  a  high  factor  9  score  reflects  high  DC-8  payloads  and 
more  timely  deliveries. 

-  Insight:  This  is  the  first  of  two  timeliness  performance  indices  for  this  scenario, 
both  of  which  relate  timeliness  of  cargo  deliveries  to  civilian  aircraft  performance. 
For  large  scale  deployments,  such  as  the  one  modeled  in  this  scenario,  the  civil 
reserve  air  fleet  (CRAF)  is  critical  to  success  in  delivering  cargo  on-time.  In  this 
case,  the  positive  correlation  between  DC-8  payloads  and  timeliness  of  deliveries 
makes  some  sense. 

Factor  10.  B-747 Payload  and  Lateness  Index. 

-  Significance:  In  general,  a  high  factor  10  score  reflects  greater  B-747  payloads 
and  more  late  deliveries. 
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-  Insight:  This  is  the  second  of  two  timeliness  indices  that  point  toward  variations 
in  CRAF  payloads  as  the  main  reason  for  variations  in  timelines  or  lateness.  In  this 
case,  though,  B-747  payloads  alone  appear  to  be  positively  correlated  with 
increased  lateness  of  deliveries.  This  relationship  was  unexpected  because  the 
B-747  is  normally  a  very  efficient  long-range  cargo  aircraft.  This  relationship  is  so 
counterintuitive  that  it  could  lead  to  suspecting  the  methodology  of  interpretation, 
however,  possible  tradeoffs  between  payload  and  fuel  as  a  result  of  routing 
decisions  provided  the  best  explanation  (Hagin,  1996).  More  fuel  and  less  payload 
would  allow  the  B-747  to  fly  fiirther  and  more  directly  without  stopping,  possibly 
increasing  the  rate  of  cargo  deliveries  and  improving  the  overall  timeliness  of  the 
deliveries. 

For  this  scenario,  the  factor  by  factor  plots  were  more  in  line  with  the  multivariate 
normality  expectation  discussed  earlier  in  the  small  scenario  random  seed  analysis.  The 
first  two  factors  are  plotted  in  Figure  4-3,  and  present  a  more  normal  bivariate  distribution 
than  was  seen  in  Figure  4-1 .  A  similar  plot  resulted  when  selecting  any  two  factors  from 
the  large  scenario  random  seed  runs. 


C-5  and  Cargo  Throughput  Index  Score 


Figure  4-3.  Plot  of  Factor  2  Scores  by  Factor  1  Scores  (Large  Scenario) 
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Potential  for  Verification  and  Validation.  As  mentioned  in  Chapter  11,  the  factor 
analysis  model  attempts  to  account  for  variation  in  the  original  data  by  identifying 
common  factors  that  cause  the  variation.  By  changing  only  the  initial  random  seed  from 
run  to  run,  the  cause,  or  source,  of  the  variation  in  output  variables  is  attributable  only  to 
the  simulation’s  internal  stochastic  processes  that  depend  on  the  initial  random  seed. 
Therefore,  the  factor  model  should  reflect  the  model’s  internal  stochastic  processes. 

In  this  case,  the  factor  scores  reflected  the  result  of  the  stochastic  mission 
execution  process  in  the  AFM.  When  one  aircraft  type  drew  shorter  ground  times,  it  got 
scheduled  more  frequently,  delivered  more  cargo,  and  used  more  MOG  resources.  In  the 
small  scenario,  an  almost  chaotic  result  occurred  due  to  small  changes  in  the  types  of 
mrcraft  that  made  more  frequent  deliveries.  In  general,  from  the  verification  standpoint, 
the  model  seems  to  be  working  as  designed.  The  exception,  of  course,  being  the  trivial 
load  collection  heuristic  and  its  relation  to  the  mission  scheduling  heuristic. 

Variable  relationships  were  also  interpreted  from  the  factor  loadings  matrix  in 
terms  of  processes  that  could  occur  in  the  real  world  airlift  system.  Though  there  was  no 
direct  comparison  to  actual  airlift  system  processes  during  execution  of  similar  TPFDDs, 
there  seems  to  be  some  potential  here  for  model  validation,  as  well. 

.Summary  of  Pandnm  Seed  Runs.  AFM’s  internal  random  seed  affected  the 
variance  of  AFM  output  variables.  Variation  for  most  AFM  output  variables  was  reflected 
in  coefficients  of  variation,  which  did  not  normally  exceed  3%  of  the  mean  values.  Some 
variables  in  the  small  scenario  exhibited  a  tendency  toward  a  bi-modal  distribution. 
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Factor  analysis  was  helpful  in  reducing  dimensionality  of  the  data  and  in  gaining 
insight  into  how  AFM  output  was  affected  by  changing  the  random  seed.  Factor  loadings 
matrices  indicated  how  AFM’s  stochastic  processes  had  independent  affects  on 
performance  characteristics  of  the  individual  aircraft  types.  In  turn,  overall  characteristics 
of  the  airlift  system  itself  seemed  to  be  a  nearly  linear  function  of  performance 
characteristics  of  the  individual  aircraft  types.  Competition  for  resources  and  cargo  were 
also  reflected,  when  significant  to  changes  in  performance.  Interpretations  of  the  factor 
loadings  matrix  tended  to  mirror  real  world  processes,  'with  the  exception  of  the  bi-modal 
problem  noted  in  the  small  scenario.  This  reflection  of  real  world  processes  seemed  to 
further  validate  AFM  as  a  useful  airlift  model. 

The  investigation  into  random  seed  effects  basically  treated  AFM  as  a  black  box, 
and  focused  on  AFM  output  variables  using  factor  analysis  to  characterize  the  inner 
workings  of  the  black  box.  In  this  sense,  this  application  of  factor  analysis  to  characterize 
a  stochastic  simulation  demonstrated  potential  for  model  verification  and  validation. 
Perturbation  Run  Data 

Perturbations  to  the  two  original  files  63  were  accomplished  using  the  FORTRAN 
program  listed  in  Appendbc  D.  This  FORTRAN  code  was  used  to  generate  ten  file  63 
perturbations  for  each  of  the  design  points  1  through  7,  as  discussed  in  Chapter  in  (see 
Table  3-1).  Ten  of  the  50  random  seed  runs  fi'om  the  previous  section  were  used  for 
design  point  0.  As  mentioned  in  Chapter  ffl,  during  the  development  and  testing  of  the 
FORTRAN  code,  test  runs  were  made  that  generated  additional  useful  perturbations  of 
the  file  63.  Because  these  file  63’ s  were  available,  they  were  included  in  the  design  as 
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design  points  8  and  9.  With  10  runs  at  each  design  point,  output  from  100  AFM  runs  was 
analyzed  during  this  portion  of  the  research.  After  computing  the  factor  scores  for  each 
run,  plotting  each  of  the  100  runs  on  factor  by  factor  plots  provided  a  graphical  tool  for 
sensitivity  analysis. 

Small  Scenario  Appendix  E  contains  the  raw  AFM  output  data  for  the  100  runs 
used.  Using  the  S AS  procedure,  PROC  FACTOR  with  a  VARIMAX  rotation,  on  this 
data  set  resulted  in  the  factor  loadings  matrix  presented  in  Table  4-7. 


Table  4-7.  Small  Scenario  Factor  Loadings  Matrix  for  the  Experimental  Design. 


Factor 

1 

MM 

C-5A  use  rate 

C-5A  average  payload 

C-5A  MTMs/aircraft/day 

BSBI 

■*»»! 

0.107 

C-141B  use  rate 

-0.889 

g«Fr«n 

0.278 

C-141B  g-cycle  time 

BtWgl 

C-141B  average  payload 

-0.881 

C-141B  MTMs/aircraft/d 

-0.914 

0.014 

C-17  use  rate 

-0.216 

-0.141 

C-1 7  g-cycle  time 

BiTtgItl 

-0.064 

-0.035 

C-17  average  payload 

0.459 

-0.8011 

-0.082 

0.188 

C-17  MTMs/aircraft/day 

0.631 

KC-10  use  rate 

0.062 

KC-10  average  payload 

-0.586 

■jssi] 

-0.011 

KC-10  MTMs/aircraft/da 

0.292 

0.121 

B-747P  use  rate 

■ZiQ] 

Outsize  tons  delivered 

0.041 

Passengers  delivered 

Total  cargo  tons  deliv 

-0.140 

KiEQ 

%  Shipments  ontime 

-0.350 

-0.290 

1  -0.810 

-0.040 

Average  days  late 

0.432 

0.066 

0.008 

Eigenvalues 

9.4591 

2.9796 

2.3861 

1.7092 

Common  Variance  (%) 

57.2% 

18.0% 

14.4% 

10.3% 

The  eigenvalues  and  their  corresponding  percentages  of  the  common  variance  are 


also  listed  in  Table  4-7.  Factor  1  was,  by  a  large  margin,  the  most  significant  of  the 
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factors.  It  was  interesting  to  note  how  the  loadings  on  the  four  factors  separated  the 
throughput  variables  into  three  categories;  outsize  cargo,  passengers,  and  total  cargo. 
These  three  categories  provided  a  logical  basis  for  interpretation  of  the  data: 


Factor  1.  Widebody  and  Passenger  Index. 

-  Significance:  In  general,  high  factor  1  scores  reflect:  high  passenger 
throughput;  high  C-5,  KC-10,  and  B-747P  use/performance;  and  low  C-141B 
use/performance. 

-  Insight:  Variance  in  C-5  and  B-747P  use  rates  accounted  for  most  of  the 
variance  in  passenger  throughput.  C-141  and  C-17  ground  cycle  times 
understandably  showed  a  corresponding  increase  when  C-5  and  B-747P  aircraft 
were  used  more,  due  to  competition  for  MOG  resources.  The  KC-10,  though  not 
a  passenger  carrying  aircraft,  loaded  highly  on  this  factor  also,  primarily  because  its 
preference  for  bulk  cargo  complemented  the  cargo  and  passenger  preferences  of 
the  C-5  and  B-747P  (see  Table  2-2).  The  contrast  between  C-5  payload  and  both 
the  C-141B  and  KC-10  payloads  probably  reflect  competition  for  oversize  cargo. 

Factor  2.  Outsize  Cargo  Index 

-  Significance:  In  general,  high  factor  2  scores  reflect  high  outsize  cargo 
throughput  and  high  C-17  use  with  less  efficient  payloads. 

-  Insight:  Variations  in  outsize  cargo  throughput  performance  was  accounted  for 
by  C-17  usage.  The  negative  loading  for  C-5  payload  reflected  competition  with 
the  C-17  for  outsize  cargo.  The  negative  loading  for  C-17  payload  indicated  less 
efficient  use  of  C-17  payload  capacity  when  carrying  outsize  cargo. 

Factor!.  Lateness  Index. 

-  Significance:  In  general,  high  factor  3  scores  reflect  more  late  deliveries. 

-  Insight:  Once  again,  the  timeliness  performance  measures  are  distinctly 
independent  of  the  other  performance  measures. 

Factor  4.  Total  Cargo  Index. 

-  Significance:  In  general,  high  factor  3  scores  reflect  high  total  cargo  throughput 
and  high  C-17  MTM/D  throughput. 
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-  Insight.  Variations  in  total  cargo  throughput  performance  was  accounted  for  by 
variations  in  C-17  MTM/D  throughput  performance.  Since  factor  2  accounted  for 
the  outsize  cargo,  this  factor  probably  reflects  the  variation  in  the  remaining  cargo, 
including  only  oversize  and  bulk  cargoes. 

Combined,  factors  2  and  4  reflect  the  small  scenario’s  preference  for  the  C-17 
aircraft’s  small  MOG  requirement  and  its  versatility  with  all  types  of  cargo.  This  was 
expected,  because  the  limited  MOG  available  among  the  seven  primary  APOD’s  makes 
this  scenario  sensitive  to  the  smaller  aircraft  types. 

The  sensitivity  of  the  small  scenario  to  the  various  perturbations  is  graphically 
depicted  in  Figures  4-4, 4-5,  and  4-6.  These  “sensitivity”  plots  were  created  by  SAS  and 
show  factor  1  scores  plotted  against  factors  2,  3,  and  4  scores,  respectively.  Each  plotted 
symbol  reflects  the  design  point  identification  code  for  one  run  (see  Table  3-1). 

When  interpreting  the  sensitivity  plots,  use  the  cluster  of  O’s  as  a  reference  point. 
The  size  of  the  cluster  of  O’s  represents,  in  a  graphical  way,  the  AFM  internal  noise,  or 
variance  that  was  expected  from  changing  the  random  seed.  For  this  analysis,  this  internal 
random  noise  was  assumed  to  be  fairly  constant  throughout  the  experimental  design  space. 
Design  points  (such  as  4,  5,  6,  7,  and  9)  with  a  cluster  larger  in  size  than  the  cluster  of  O’s 
represent  an  increase  in  performance  variance  due  to  the  perturbation  effects.  Relative 
increases  or  decreases  in  mean  airlift  system  performance  are  reflected  in  the  lateral 
displacements  of  the  clusters  from  each  other.  These  lateral  displacements  reflect  a 
change  in  the  scenario’s  mean  performance  due  to  the  perturbations.  The  following 
discussions  of  the  effects  of  perturbations  refers  to  the  magnitude  of  these  lateral 
displacement  as  the  amount  ol performance  mean  sensitivity  to  the  perturbations. 
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Figure  4-4.  Sensitivity  Plot  of  Widebody  and  Passenger  Index  vs.  Outsize  Cargo  Index  (Small  Scenario) 
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Figure  4-5.  Sensitivity  Plot  of  Widebody  and  Passenger  Index  vs.  Lateness  Index  (Small  Scenario) 
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The  performance  variation,  as  measured  by  all  four  performance  indices, 
significantly  increased  with  the  APOE/APOD  perturbations  (design  points  4,  5,  6,  7,  and 
9).  The  performance  mean  sensitivity  to  APOE/APOD  perturbations  was  fairly  significant 
for  the  Passenger  Throughput  Index,  but  not  significant  for  the  other  indices.  Figure  4-4 
seems  to  indicate  a  possible  interaction  or  tradeoff  between  passenger  throughput  and 
lateness,  as  seen  by  the  diagonal  alignment  of  the  4,  5,  6,  7,  and  9  design  points.  This 
interaction  between  passengers  and  lateness  was  also  noted  as  a  characteristic  of  this 
scenario  during  the  analysis  of  the  random  seed  runs  (See  the  discussion  of  Factor  4  fi*om 
Table  4-6).  Though  the  APOE/APOD  perturbation  scheme  was  very  simplistic,  Figures 
4-4  and  4-6  imply  that  it  is  possible  to  improve  passenger  throughput  for  this  scenario, 
without  sacrificing  cargo  throughput,  by  using  different  onload  and  offload  locations. 

The  performance  variation  and  performance  mean  sensitivity  to  ALD/RDD 
perturbations  (design  points  2,  3,  6,  7,  and  8)  were  not  significant  for  any  of  the  indices. 
The  implication  was  that  passenger  and  cargo  throughput,  in  terms  of  numbers  of  people 
and  cargo  tonnage,  remained  fairly  constant,  despite  changes  to  the  order  of  the 
requirement  lines  in  the  TPFDD  and  changes  of  no  more  than  one  day  to  the  ALD  and 
RDD  timeline.  Another  implication  was  that  aggregate  measures  of  timeliness  (or 
lateness),  also  remained  fairly  constant.  The  two  timeliness  performance  variables  are 
considered  aggregate  measures  of  timeliness  in  that  they  reflect  the  timeliness  of  all 
delivered  shipments. 

The  performance  variation  due  to  the  10  percent  load  strain  perturbations  (design 
points  1,  3,  5,  and  7)  was  not  significant  for  any  of  the  indices.  The  performance  mean 
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sensitivity  to  load  strain  perturbations  was  mildly  significant  only  for  the  Total  Cargo 
Throughput  Index  and  possibly  the  Outsize  Cargo  Throughput  Index.  In  Figure  4-6,  the 
lack  of  a  significant  vertical  displacement  between  the  cluster  of  O’s  and  the  clusters  of  1  s 
and  3’s  indicates  that  the  load  strain  perturbations,  which  increased  passenger 
requirements  10%,  have  little  impact  on  the  Passenger  Throughput  Index.  Figure  4-6 
shows  most  of  the  1,  3,  5,  and  7  plots  on  the  right  side  of  the  plot,  possibly  indicating  a 
mild  increase  in  total  cargo  throughput.  Whether  or  not  this  increase  is  statistically 
significant,  this  scenario  cannot  handle  much  of  an  increase  in  demand  for  passenger  or 
cargo  traffic.  The  performance  mean  sensitivity  of  outsize  cargo  throughput,  as  seen  by 
the  horizontal  displacements  in  Figure  4-4,  appear  insignificant  except  for  the  slight 
negative  horizontal  displacement  between  the  clusters  of  I’s  and  2’s.  The  only  notable 
difference  between  design  points  1  and  2  is  the  change  in  the  proportions  of  outsize  and 
oversize  cargo  (ALD/RDD  effects  were  insignificant).  Outsize  cargo  throughput  seemed 
indifferent  to  the  load  strain  perturbation  until  the  relative  proportions  between  the  cargo 
classes  changed,  implying  that  the  relative  proportions  between  the  cargo  classes  can 
impact  the  significance  of  effects  caused  by  straining  the  system. 

The  performance  variation  due  to  perturbing  the  relative  proportions  of  outsize 
and  oversize  cargo  (design  points  1,  2, 4,  and  7)  was  not  significant.  However,  the 
performance  mean  sensitivity  to  perturbing  the  relative  proportions  of  outsize  and 
oversize  cargo  was  mildly  significant  for  the  Lateness  Index  and  very  significant  fisr  the 
Outsize  Throughput  Index.  Figure  4-5  shows  a  small  relative  increase  in  lateness 
performance.  Figure  4-4  shows  a  large  relative  increase  in  outsize  cargo  throughput 
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performance.  Since  C-17  use  rate  also  loads  fairly  high  on  factor  2,  the  increase  in  outsize 
cargo  throughput  is  most  likely  due  to  the  C-17’s  capability  to  deliver  outsize  cargo  into 
small  airfields.  Though  there  was  an  increase  in  outsize  cargo  performance,  there  was  no 
discernible  change  in  total  cargo  throughput  performance,  as  shown  in  Figure  4-6, 
indicating  the  consistency  of  the  total  cargo  throughput  measure. 

It  is  interesting  to  note,  across  all  four  perturbation  schemes,  how  the  total  cargo 
throughput  index  was  relatively  unaffected,  reflecting  the  consistency  and  robustness  of 
the  total  cargo  tons  delivered  variable.  The  robustness  of  this  variable  was  also  indicated 
in  the  analysis  of  the  large  scenario. 

T  .arpft  Srp.narin  Appendk  F  contains  the  raw  AFM  output  data  for  the  100  runs 
used  during  this  portion  of  the  research.  Using  the  SAS  procedure,  PROC  FACTOR  with 
a  VARIMAX  rotation,  on  this  set  of  100  runs  resulted  in  the  factor  loadings  matrix 
presented  in  Table  4-8.  AFM  did  not  reliably  output  C-17  ground  cycle  statistics  for  all 
design  points,  so  the  variable  was  discarded,  leaving  30  variables  for  fector  analysis. 

The  eigenvalues  and  their  corresponding  percentages  of  the  common  variance  are 
also  listed  in  Table  4-8.  Factor  1  was,  by  a  large  margin,  the  most  significant  of  the  eight 
factors.  It  was  interesting  to  note  how  the  three  throughput  categories,  outsize  cargo, 
passengers,  and  total  cargo,  separated  again  onto  three  different  factors. 
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Table  4-8.  Large  Scenario  Factor  Loadings  Matrix  for  the  Experimental  Design 


Factor 

1 

2 

3 

4 

5 

mm 

C-5A  use  rate 

0.S88 

-0.030 

-0.249 

0.353 

0.065 

Qg;g| 

C-5A  g-cycle  time 

-0.826 

0.003 

C-5A  average  payload 

0.140 

C-5A  MTMs/aircraft/day 

■»ygiai 

0.043 

-0.001 

0.100 

0.306 

C-141B  use  rate 

QQQ 

mwi^ 

0.092 

0.203 

-0.182 

-0.160 

C-1 41 B  g-cycle  time 

-0.6921 

0.032 

-0.233 

0.298 

0.329 

C-141B  average  payload 

0.197 

-0.129 

0.198 

0.030 

-0.616 

C-1 7  use  rate 

0.7141 

0.112 

C-1 7  average  payload 

0.90^ 

022^ 

C-1 7  MTMs/aircrafl/day 

KEQ] 

0.001 

-0.090 

0.334 

mEES 

EBS 

KC-10  use  rate 

0.113 

0.083 

-0.415 

0.252 

0.007 

qees 

KC-10  g-cycle  time 

-0.876 

0.034 

0.031 

0.267 

-0.049 

0.003 

0.241 

-0.077 

KC-10  average  payload 

-0.130 

0.033 

0.857 

0.213 

-0.091 

-0.138 

-0.101 

0.110 

KC-10  MTMs/aircrafl/day 

0.317 

0.026 

0.847 

0.055 

0.048 

-0.165 

-0.134 

-0.057 

B-747  use  rate 

0.154 

-0.054 

0.314 

-0.007 

0.061 

0.074 

B-747  g-cycle  time 

0.187 

-0.006 

0.155 

-0.006 

0.173 

-0.240 

-0.090 

0.761 

B-747  average  payload 

-0.210 

0.031 

-0.517 

-0.082 

0.065 

0.062 

B-747  MTMs/aircrafl/day 

-0.0361 

0.099 

0.082 

0.282 

-0.045 

0.071 

0.051 

B-747P  use  rate 

0.681 

-0.090 

-0.048 

-0.023 

0.234 

-0.232 

0.220 

0.131 

B-747P  g-cycle  time 

0.468 

0.026 

0.247 

-0.021 

0.611 

-0.090 

0.108 

0.151 

DC-8  use  rate 

-0.177 

0.029 

-0.063 

0.050 

0.006 

0.893 

0.100 

-0.179 

DC-8  g-cycle  time 

-0.002 

0.048 

0.200 

-0.241 

0.251 

-0.835 

0.102 

0.077 

DC-8  average  payload 

0.190 

-0.094 

-0.165 

-0.222 

-0.045 

-0.008 

0.778 

-0.089 

DC-8  MTMs/aircraft/day 

0.919 

-0.038 

0.118 

-0.034 

0.220 

-0.016 

0.126 

0.089 

Outsize  tons  delivered 

-0.090 

-0.777 

0.023 

-0.186 

0.236 

0.032 

-0.160 

-0.060 

Passengers  delivered 

-0.577 

0.158 

-0.094 

0.165 

-0.103 

Total  cargo  tons  delivered 

IQQ] 

-0.117 

BSSlUii 

%  Shipments  ontime 

-0.500 

0.189 

-0.006 

-0.041 

1-0.744 

0.120 

Average  days  late 

0.439 

0.037 

-0.060 

0.104 

-0.161 

-0.127 

0.046 

Eigenvalues 

7.689 

2.607" 

2.586 

2.574 

2.484 

1.903 

1.679 

1.654 

Common  Variance  (%) 

11.3% 

11.2% 

11.1% 

10.7% 

8.2% 

7.2% 

7.1% 

Factor  1.  Total  Cargo  Index. 

-  Significance:  In  general,  high  factor  1  scores  reflect  high  cargo  throughput  and 
high  aircraft  performance. 

-  Insight:  Except  for  the  C-141B  and  the  DC-8,  all  aircraft  use  rates  were 
positively  correlated  with  cargo  throughput.  Though  not  contributing  to  total 
cargo  throughput,  B-747P  (passenger  version)  aircraft  also  loaded  highly  upon  this 
first  factor,  providing  part  of  the  passenger  dimension  to  this  factor  as  well.  The 
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exclusion  of  the  smaller  C-141B  and  DC-8  aircraft  from  this  first  factor  reflects  the 
scenario’s  preference  for  the  larger,  longer  range  aircraft. 

Factor  2.  C-1 7  and  Outsize  Cargo  Contrast. 

-  Significance:  In  general,  high  factor  2  scores  reflect  high  C-1 7  throughput 
performance  and  low  outsize  cargo  throughput.  Low  factor  2  scores  reflect  more 
outsize  cargo  throughput,  but  less  efficient  use  of  C-1 7  payload  capacity. 

-  Insight:  In  contrast  to  the  small  scenario,  where  C-17  throughput  and  Outsize 
throughput  were  positively  correlated,  this  scenario  shows  a  negative  correlation 
between  these  variables.  The  long  distance  between  APOE/ APOD  pairs  in  this 
scenario  most  likely  forced  a  tradeoff  between  C-17  payload  and  fuel.  Less  fuel 
and  more  payload  meant  stopping  more  often  for  fuel,  which  slowed  the  rate  of 
C-17  deliveries,  decreasing  overall  C-17  throughput.  Therefore,  increasing  C-17 
MTM/D  throu^put  resulted  from  taking  on  more  fiael  and  less  outsize  payload. 
Since  the  C-5  payload  is  positively  correlated  with  C-17  throughput  measures,  the 
same  argument  applies  to  C-5  payload  and  fuel  tradeoffs.  These  two  aircraft  types 
deliver  most  of  the  outsize  cargo  in  this  scenario,  and  the  flow  of  outsize  cargo 
seems  to  benefit  from  their  sacrificing  a  little  payload  for  extra  fuel  to  fly  further. 

Factor  3.  KC-10  and  Passenger  Contrast. 

-  Significance:  In  general,  high  factor  3  scores  indicate  high  KC-10  throughput 
performance  and  low  passenger  throughput  performance. 

-  Insight:  This  factor  also  reflects  competition  between  KC-10  and  B-747  aircraft 
for  cargo,  which  is  understandable  because  they  share  an  almost  exclusive 
preference  for  bulk  cargo.  Since  KC-10  and  B-747  aircraft  were  not  able  to  carry 
passengers,  the  passengers  delivered  variable  seemed  misplaced.  There  might  be  a 
MOG  resource  constraint  reflected  in  this  relationship,  where  larger  payloads  on 
KC-10  aircraft  caused  KC-10  aircraft  to  spend  more  time  on  the  ground  and 
limited  the  amount  of  MOG  resources  left  for  passenger  carrying  aircraft. 

Including  the  B-747P  use  rate  observations  in  Factor  1,  passenger  throughput 
performance  does  not  seem  to  be  as  clearly  separated  from  the  many  dimensions  of 
cargo  throughput  performance,  which  contrasts  with  their  distinct  independence  in 
the  small  scenario. 

Factor  4.  C-5  MTM/D  and  C-141B  Use  Rate  Index. 

-  Significance:  A  high  factor  4  score  reflects  high  C-141B  use  and  high  C-5 
MTM/D  throughput  performance. 

-  Insight:  There  was  a  positive  correlation  between  C-141B  use  rate  and  C-5 
MTM/D  throughput  performance  in  this  scenario.  Factor  8  will  complete  the 
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second  dimension  of  the  C-5  and  C-141B  relationship  in  this  scenario.  Though 
competition  for  common  payloads  (oversize  and  bulk  cargo)  is  evident  in  the 
Factor  8  loadings,  this  factor  also  possibly  reflects  a  different  dimension  of 
oversize  and  bulk  cargo  throughput. 

Factor  5.  Lateness  Index. 


-  Significance:  In  general,  a  high  factor  5  score  reflects  more  late  deliveries. 

-  Insight:  As  was  seen  in  the  random  seed  runs,  CRAF  aircraft  again  were 
identified  with  timeliness  performance,  indicating  the  importance  of  CRAF  aircraft 
to  successful  timeliness  performance. 

Factor  6.  DC-8  Use  Rate  Index. 

-  Significance:  In  general,  a  high  factor  6  score  reflects  high  DC-8  use  and  low 
DC-8  ground  times. 

-  Insight:  This  factor  directly  reflects  the  amount  of  flying  performed  by  the  DC-8 
fleet  in  this  scenario. 

Factor?.  CRAF  Payload  Index. 

-  Significance:  In  general,  a  high  factor  7  score  reflects  high  DC-8  and  B-747 
payloads,  which  together  add  up  to  total  CRAF  cargo  payload  capacity. 

-  Insight:  This  factor  indicates  a  positive  correlation  between  DC-8  and  B-747 
cargo  payloads.  Though  these  two  aircraft  share  a  preference  for  bulk  cargo,  there 
was  no  apparent  competition  for  payloads  between  them.  Instead  of  competing 
with  each  other.  Factor  8  loadings  indicate  that  B-747  aircraft  may  be  competing 
with  the  military  cargo  aircraft.  This  observation  combined  with  the  relationship 
between  B-747  and  DC-8  payloads  found  in  this  factor  indicate  a  competition  for 
cargo  between  CRAF  and  m^tary  aircraft.  The  reason  this  relationship  was 
spread  over  the  last  two  factors  in  independent  CRAF  and  military  dimensions 
most  likely  has  to  do  with  the  fact  that  the  military  and  civilian  aircraft  “fly” 
different  airways  and  use  a  different  airlift  system  network  structure  in  AFM. 

Factor  8.  C-5  and  C-141B  Payload  Contrast. 

-  Significance:  In  general,  a  high  factor  8  score  reflects  high  C-5  payloads,  low 
C-141B  payload,  and  longer  B-747  ground  times.  Low  factor  8  scores  reflect  low 
C-5  payloads,  high  C-141B  payloads,  and  shorter  B-747  ground  times. 

-  Insight:  Competition  between  C-5  and  C-141B  payloads  indicated  here 
complete  the  relationship  found  between  these  two  aircraft  types  in  Factor  4.  The 
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significant  loading  of  B-747  ground  times  relates  to  either  MOG  constraints  or 
competition  for  cargo,  as  previously  mentioned  in  the  Factor  7  insights. 

The  sensitivity  of  the  large  scenario  to  the  various  perturbations  can  be  seen  by 
looking  at  the  sensitivity  plots,  Figures  4-7  and  4-8.  As  before,  each  plotted  S5mibol 
reflects  the  design  point  identification  code  for  one  run  (see  Table  3-1).  There  are  many 
possible  combinations  of  the  8  factors  that  could  be  used  to  provide  other  sensitivity  plots, 
but  only  two  were  needed  to  support  the  following  discussion.  Appendbc  F  contains  the 
factor  scores  for  each  run  and  can  be  plotted  to  obtain  more  sensitivity  plots,  if  desired. 

When  interpreting  the  sensitivity  plots,  use  the  cluster  of  O’s  as  a  reference  point. 
For  this  analysis,  internal  random  noise  was  assumed  to  be  constant  throughout  the 
experimental  design  space.  The  size  of  the  cluster  of  O’s  represents,  in  a  graphical  way, 
the  AFM  internal  noise,  or  variance  that  was  expected  firom  changing  the  random  seed. 
Design  points  (such  as  4,  5,  6,  7,  and  9)  with  a  cluster  larger  in  size  than  the  cluster  of  O’s 
represent  an  increase  in  performance  variance  due  to  the  perturbation  effects.  Relative 
increases  or  decreases  in  mean  airlift  system  performance  is  reflected  in  the  lateral 
displacement  of  the  clusters  fi-om  each  other.  The  relative  displacements  of  the  clusters 
reflect  the  performance  mean  sensitivity  to  the  perturbations. 
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Figure  4-7.  Sensitivity  Plot  of  Total  Cargo  Index  vs.  C-17  and  Outsize  Cargo  Contrast  (Large  Scenario) 
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Figure  4-8.  Sensitivity  Plot  of  Total  Cargo  Index  vs.  Lateness  Index  (Large  Scenario) 


The  performance  variation,  as  measured  by  all  eight  performance  indices, 
significantly  increased  with  the  APOE/APOD  perturbations  (design  points  4,  5,  6,  7,  and 
9).  The  performance  sensitivity  to  APOE/APOD  perturbations  was  significant  for  the 
Total  Cargo  Throughput  Index,  but  not  significant  for  the  other  indices.  The  negative 
vertical  displacement  of  these  design  points  relative  to  the  other  design  points  in 
Figure  4-7  implies  that  randomly  changing  APOE/APOD  pairs  decreased  total  cargo 
throughput  performance  and/or  aircraft  use  rates.  However,  looking  at  the  cluster  of  9’s 
versus  the  cluster  of  O’s  in  Figure  4-8  reveals  a  positive  impact  on  timeliness. 

The  performance  variation  and  performance  sensitivity  to  ALD/RDD 
perturbations  (design  points  2,  3,  6,  7,  and  8)  were  not  significant  for  any  of  the  indices. 
The  implication  was  that  airlift  system  performance,  by  all  measures,  remained  fairly 
constant,  despite  changes  to  the  order  of  the  requirement  lines  in  the  TPFDD  and  changes 
of  no  more  than  one  day  to  the  ALD  and  RDD  timeline. 

Th&  performance  variation  and  performance  sensitivity  due  to  the  10  percent  load 
strain  perturbations  (design  points  1,  3,  5,  and  7)  were  not  significant  for  any  of  the 
indices.  The  obvious  implication  fi-om  this  result  was  that  this  scenario  was  tasking  the 
airlift  system  to  capacity  and  it  could  not  handle  any  increase  in  demand  for  passenger 
or  cargo  traffic. 

The  performance  variation  due  to  perturbing  the  relative  proportions  of  outsize 
and  oversize  cargo  (design  points  1,  2, 4,  and  7)  was  not  significant.  However,  the 
performance  sensitivity  to  perturbing  the  relative  proportions  of  outsize  and  oversize 
cargo  was  mildly  significant  for  the  C-17  and  Outsize  Cargo  Contrast  (See  Figure  4-7). 
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Figure  4-7  shows  the  small  negative  displacement  of  the  I’s  and  2’s  relative  to  the  O’s  and 
3’s,  indicating  a  slight  increase  in  outsize  throughput  performance.  Though  there  was  a 
slight  increase  in  outsize  cargo  throughput  performance,  there  was  no  discernible 
change  in  total  cargo  throughput  performance,  as  shown  in  the  vertical  dimension  of 
Figure  4-7,  indicating  the  consistency  of  the  total  cargo  throughput  measure  once  again. 

Summary  nf  Perturbation  Runs.  The  application  of  factor  analysis  as  a  sensitivity 
analysis  tool  was  demonstrated  and  seemed  useful  in  identifying  the  effects  of  the  four 
perturbation  schemes  used.  The  sensitivity  plots  graphically  depicted  the  effects  of  the 
combination  of  perturbations  included  in  each  experimental  design  point.  These  sensitivity 
plots  indicated  that  airlift  system  performance,  as  measured  by  AFM  output,  was  affected 
by  perturbations  to  the  TPFDD.  In  the  small  scenario,  the  perturbation  schemes  seemed 
to  have  a  greater  effect  than  in  the  large  scenario,  where  the  airlift  system  was  tasked  to 
capacity. 

In  both  scenarios,  total  cargo  throughput  seemed  to  be  robust  and  consistent, 
relatively  unaffected  across  all  perturbation  schemes.  This  is  particularly  noteworthy, 
because  it  puts  in  perspective  the  frequent  historical  use  of  total  tons  delivered  as  an 
indicator  of  overall  airlift  system  performance.  For  measuring  overall  cargo  throughput, 
there  can  obviously  be  many  variations  of  the  TPFDD  and  this  statistic  would  still  be  a 
reasonable  measure  of  how  many  tons  could  be  delivered.  However,  to  use  this  variable 
as  the  sole  measure  of  airlift  system  performance  would  ignore  pertinent  information  on 
the  many  other  aspects  of  airlift  performance. 
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Perturbing  APOE/APOD  pairs  significantly  affected  variance  of  most  performance 
measures.  Though  the  coded  algorithm  may  not  have  been  a  very  realistic  approach  to 
r.hf^ngit^g  APOE/APOD  pairs,  the  results  fi-om  blindly  changing  APOE/APOD  pairs  had 
significant  consequences  for  some  major  indices  of  airlift  system  performance.  In  the 
small  scenario,  passenger  throughput  significantly  increased,  as  expected,  by  redistributing 
the  passengers  amongst  the  destination  airports.  For  the  same  reasons  passenger 
throughput  was  expected  to  increase  in  the  small  scenario,  cargo  throughput  was  expected 
to  increase  in  the  large  scenario.  However,  cargo  throughput  significantly  decreased  as  a 
result  of  this  perturbation  scheme.  The  unpredictability  and  high  sensitivity  of  airlift 
system  performance  to  this  perturbation  scheme  seemed  almost  chaotic,  where  relatively 
small  changes  to  input  caused  large  changes  in  output  (Bauer,  1996).  This  high 
sensitivity  to  APOE/APOD  changes  could  have  also  been  a  result  of  too  large  a 
perturbation.  Allowing  a  maximum  of  a  ten  percent  change  in  distance  between  APOE 
and  APOD  for  each  line  of  a  TPFDD  could  have  been  too  much.  It  is  possible  that  smaller 
perturbations  might  have  had  a  more  stable  and  predictable  effect  on  AFM  output. 
Whether  the  result  is  due  to  chaos  or  too  large  a  perturbation,  AFM  output  subject  to 
completely  unconstrained  random  TPFDD  generation,  with  respect  to  the  APOEs  and 
APODs,  would  most  likely  be  as  sensitive  and  unpredictable  and  is  not  recommended. 

Perturbing  available-to-load  dates  (ALD’s)  and  required  delivery  dates  (RDD’s), 
by  plus  or  minus  one  day,  and  re-sorting  the  TPFDD  by  RDD  then  AID,  impacted  the 
order  in  which  the  requirements  were  processed  by  AFM.  Though  the  requirements  were 
processed  in  a  different  order  and  the  timeline  was  slightly  disturbed,  AFM  output  was 
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relatively  unaffected  by  this  perturbation  scheme.  The  major  implication  to  be  highlighted 
from  this  result  is  that  the  relative  order  and  priority  of  the  individual  TPFDD  requirement 
lines  have  no  significant  impact  on  aircraft  usage  or  total  system  throughput  measures. 
Other  aspects  of  airlift  system  performance,  like  combat  unit  closure,  might  be  affected, 
but  were  not  considered  during  this  research. 

Increasing  all  cargo  and  passenger  requirements  by  10%  to  strain  the  airlift  system, 
effected  increased  performance  in  the  small  scenario,  where  the  airlift  system  had  more 
unused  capacity;  but  it  had  little  impact  on  the  large  scenario.  Knowing  that  most 
TPFDDs  start  out  transportationally  impossible  and  that  cargo  and  passenger  delivery 
timelines  for  most  TPFDDs  are  relaxed  until  they  are  transportationally  feasible,  this 
perturbation  seemed  more  of  a  curiosity  than  a  useful  endeavor.  However,  this 
perturbation  was  useful  in  identifying  what  areas  of  airlift  performance  had  unused 
capacity  still  available  and  what  areas  didn’t. 

As  mentioned  in  Chapter  H,  possible  changes  to  national  pre-positioning  strategy 
would  cause  a  shift  in  the  amounts  of  outsize  and  oversize  cargoes  within  the  TPFDD 
requirements.  Perturbing  the  TPFDDs  to  reflect  this  change,  in  favor  of  increasing  outsize 
at  the  expense  of  oversize  cargo,  did  not  seem  to  affect  total  cargo  tons  delivered  in  either 
scenario.  In  general,  an  increase  in  outsize  cargo  delivered  was  noted,  but  not  always.  In 
the  small  scenario,  when  combining  this  perturbation  with  the  load  strain  perturbation, 
outsize  cargo  delivered  was  less  than  with  the  load  strain  perturbation  alone. 
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Summary 


In  light  of  the  problem  statement  and  research  objectives  mentioned  in  Chapter  I, 
the  following  general  methodology  for  conducting  airlift  system  performance  analysis  with 
perturbed  TPPDDs  was  demonstrated: 

1)  Identify  AFM  output  variables  reflecting  desired  MOE’s 

2)  Analyze  sensitivities  to  AFM  random  seed  using  factor  analysis 

3)  Define  perturbation  schemes 

4)  Analyze  sensitivities  to  perturbations  using  factor  analysis 

These  general  steps  were  used  to  identify  the  relative  effects  of  four  TPFDD  perturbation 
schemes  on  AFM  output  variables,  indicating  that  airlift  system  performance  was  affected. 

In  addition  to  demonstrating  the  above  methodology,  the  results  from  the  random 
seed  run  data  provide  the  first  known  documentation  on  the  effects  of  changing  the  AFM 
simulation’s  initial  random  seed.  A  promising  application  of  fector  analysis  to  large, 
stochastic  simulation  model  verification  and  validation  was  also  demonstrated.  A  potential 
quirk  in  the  trivial  load  portion  of  the  mission  scheduling  heuristic  was  identified  during 
this  portion  of  the  analysis. 

During  analysis  of  the  perturbation  run  data,  sensitivity  plots  provided  a  nice 
graphical  tool  for  conducting  sensitivity  analysis  for  complex  stochastic  processes.  The 
sensitivity  plots  were  a  result  of  plotting  the  resulting  factor  scores  for  each  run  on 
separate  axes.  The  plotted  points  were  coded  point  with  a  symbol  reflecting  the  specific 
perturbation  schemes  tested.  The  effects  of  the  perturbations  on  mean  performance  and 
on  the  associated  variance  of  that  performance  were  easly  identified  from  the  sensitivity 
plots. 
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V.  Conclusions  and  Recommendations 


Introduction 

This  chapter  provides  a  summary  of  conclusions  and  recommendations  resulting 
from  this  thesis  effort.  At  the  end  of  the  chapter,  some  suggestions  for  possible  future 
research  are  also  provided. 

Conclusions 

This  thesis  was  an  exploratory  look  into  the  effects  of  a  varying  TPFDD  on  airlift 
system  performance.  The  3-step  general  methodology,  as  outlined  and  demonstrated  in 
this  thesis,  provided  an  approach  to  analyzing  airlift  system  performance  across  several 
similar  TPFDDs  by  perturbing  a  given  TPFDD.  In  general,  airlift  system  performance  was 
affected,  and  sometimes  significantly,  by  the  perturbation  schemes  used.  The  high  degree 
of  sensitivity  with  respect  to  the  locations  specified  in  the  TPFDD  seemed  to  counter 
arguments  supporting  the  usefulness  of  unconstrained  random  TPFDD  generation  in  airlift 
analysis. 

Total  cargo  tons  delivered  was  fairly  insensitive  to  all  perturbation  schemes, 
indicating  the  robustness  and  consistency  of  this  airlift  system  performance  measure.  This 
observation  points  out  the  strength  and  weakness  of  this  measure  as  the  sole  indicator  of 
airlift  system  performance.  The  strength  lies  in  the  fact  that  there  could  be  many 
variations  of  a  TPFDD,  or  errors  in  a  TPFDD,  and  this  measure  would  reflect  the  total 
throughput  capacity  of  the  airlift  system.  The  weakness  lies  in  the  lack  of  information  on 
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the  many  other  aspects  of  airlift  system  performance.  In  particular,  timeliness  of 
deliveries,  another  key  aspect  of  airlift  system  performance,  is  not  at  all  reflected  or 
implied  by  the  total  cargo  throughput  measure.  So,  as  a  measure  of  raw  mrlift  system 
capacity,  total  cargo  tons  delivered  seems  to  be  a  fairly  useful  and  robust  statistic,  being 
fairly  insensitive  to  many  details  within  the  TPFDD.  But  as  a  measure  of  how  well  the 
airlift  system  performed,  as  a  whole,  this  statistic  lacks  the  information  required  to  make 
such  an  assessment. 

This  thesis  also  provides  the  first  known  documentation  on  the  effects  of  the  AFM 
simulation’s  initial  random  seed  on  AFM  output.  Most  of  the  output  variables  collected 
for  this  research  had  a  coefficient  of  variation  less  than  3  percent. 

Factor  analysis,  as  applied  in  this  research,  was  useful  in  determining  sensitivities 
to  the  perturbations  and  in  keeping  the  size  of  the  problem  manageable.  During  analysis 
of  the  random  seed  run  data,  the  potential  for  a  new  application  of  factor  analysis  to 
stochastic  simulation  model  verification  and  validation  was  demonstrated.  During  analysis 
of  the  perturbation  run  data,  the  potential  for  another  new  application  of  factor  analysis  to 
sensitivity  analysis  of  complex  stochastic  processes  was  demonstrated.  In  particular,  the 
sensitivity  plots,  when  coded  by  the  experimental  design  point  identification  codes, 
provided  an  extremely  useful  graphical  tool  for  conducting  sensitivity  analysis. 
Recommendations 

The  AFM  mission  scheduling  heuristic  seemed  to  be  incompatible  with  the  large 
number  of  detailed  cargo  requirements  in  the  small  scenario’s  TPFDD.  Many  of  these 
detailed  requirements  were  considered  trivial  loads  by  most  of  the  modeled  aircraft  types. 
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The  AFM  simulation’s  heuristic  to  indirectly  collect  those  trivial  loads  instead  of 
dispatching  an  aircraft  to  collect  them  seemed  extremely  sensitive  to  the  frequency  with 
which  the  various  types  were  scheduled.  Though  the  effects  of  this  quirk  in  the  mission 
scheduling  heuristic  were  relatively  small  on  the  practical  level,  a  more  stable  heuristic 
should  be  investigated  to  improve  the  fidelity  of  the  AFM  simulation. 

Due  to  the  high  degree  of  sensitivity  observed  in  airlift  performance,  with  respect 
to  the  locations  specified  in  the  TPFDD,  unconstrained  random  TPFDD  generation  is  not 
recommended. 

A  perturbed  TPFDD  approach  to  airlift  system  analysis  could  provide  useful 
information  during  the  TPFDD  creation  process,  identifying  which  characteristics  of  the 
TPFDD  are  hindering  airlift  system  performance. 

Suggestions  for  Future  Research 

There  were  many  aspects  of  airlift  system  performance  that  could  be  evaluated  in 
this  research.  The  potential  exists  to  investigate  the  effects  of  TPFDD  perturbations  on 
more  focused,  detailed  performance  variables  (e.g.  crew  performance  statistics,  individual 
unit  closure,  etc.).  The  four  perturbation  schemes  used  in  this  research  are  not  the  only 
ways  to  perturb  TPFDD  data.  Therefore,  another  opportunity  exists  for  repeating  this 
research  with  a  different  set  of  perturbation  schemes. 

The  methodology  presented  in  this  thesis  also  applies  to  changes,  or  perturbations, 
in  other  inputs  to  the  airlift  system  as  well.  Effects  of  varying  MOG  resources,  MHE 
resources,  or  fleet  mixes  could  be  analyzed  in  the  same  manner. 
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There  seems  to  be  a  fairly  strong  push  to  randomly  generate  TPFDDs  for  use  in 
airlift  analysis  (Hagin,  1996).  This  capability  would  allow  for  analysis  in  situations  where 
a  formal  TPFDD  is  not  available.  For  instance,  in  Operation  Desert  Shield  we  began  the 
deployment  phase  without  an  operational  plan  or  a  feasible  transportation  plan  (Lund  and 
others,  1993  :xiv).  Without  a  TPFDD,  pre-deployment  analysis  was  limited  and  the 
resulting  policy  decisions  were  not  as  efiective  as  they  could  have  been.  The  result,  as 
documented  by  RAND,  was  ineflBcient  performance  of  the  airlift  fleet  during  the  first 
weeks  of  the  operation  (Lund  and  others,  1993  :xiv).  For  this  type  of  situation  in  the 
future,  randomly  generated  TPFDDs  could  provide  a  basis  for  airlift  performance  analysis. 
Based  on  the  results  of  this  research,  aside  from  the  aspect  of  operational  feasibility  for 
any  randomly  generated  TPFDD,  well-defined  constraints  on  certain  characteristics  of  a 
TPFDD  will  be  needed  to  keep  performance  measures  stable  enough  to  be  useful. 
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Appendix  A:  Description  of  Airlift  Flow  Module  Input  File  63 


The  following  file  description  was  extracted  fi-om  the  documentation  provided  with 
the  Airlifl:  Flow  Module  (AFM)  software. 


Input  File  63,  Requirement  Description 

USE:  The  MASS  cargo  requirement  as  derived  fi-om  a  TPFDD. 


ITEM 

LENGTH 

FORMAT 

COLUMNS 

REMARKS 

requirement_id 

5 

i5 

1  -  5 

Unique  ID  (1)(2) 

theater 

10 

alO 

6-15 

(3) 

jopes_priority 

5 

i5 

16-20 

Not  used.  Must  be  blank  or  0 

commodityjcode 

2 

i2 

21-22 

See  commodity  code  in  file  50 

available_load_date 

3 

i3 

23-25 

Day  cargo  available  to  load  (4) 

required_deUvery_date 

3 

i3 

26-28 

Day  this  cargo  must  be  delivered 
(4) 

APOE 

4 

a4 

29-32 

Onload  ICAO  (7) 

APOD 

4 

a4 

33-36 

Offload  ICAO  (7) 

direct_delivery_loc 

4 

a4 

37-40 

Direct  Delivery  Location  ICAO 
(7) 

cargojtype 

1 

al 

42-42 

S  -  strat^c  or  T  -  tactical 

amount_l 

7 

f7.1 

44-50 

Outsize  tons  (5) 

amount_2 

7 

f7.1 

51-57 

Oversize  tons 

amount_3 

7 

f7.1 

58-64 

Bulk  tons 

amount_4 

NOTES; 

7 

f7.1 

65-71 

Number  of  pax  (6) 

1 .  Must  be  grouped  together  by  theater. 

2.  Do  not  exceed  the  maximum  number  of  requirements  allowed.  Requirement_id  can  exceed  the  max 
munber  of  requirements  allowed. 

3.  The  theater  in  file  3 1  and  file  63  must  match.  Also,  all  file  3 1  theaters  must  have  at  least  one  file  63 
requirement. 

4.  When  processing  a  requirement,  sort  by  available_to_load  date,  and  then  on  the 
required_deliveiy_date  so  that  the  most  demanding  time  windows  are  at  the  top  of  the  file  because 
AIM  always  processes  FIFO.  Then  renumber  the  requirement  ids  in  sequence,  honoring  like  theaters. 
Hours  0-24  is  Day  0, ...  for  ALD  and  RDD  purposes. 

5.  For  any  given  requirement,  preferred  cargo  >  0,  total  compatible  cargo  must  exceed  an  aircraft's  trivial 
load  limit,  and  the  aircraft  must  be  allowed  to  on/off  at  the  APOE/APOD  or  the  requirement  will  close 
(see  Files  1)  for  that  aircraft  type. 

6.  The  load  amount  for  passengers  is  figured  as  tons  using  the  formula:  (amount_4  *  350.0)  /  2000.0 

7.  All  ICAOs  used  in  file  63  must  be  in  file  25,  "Airbase  List"  and  its  supporting  files. 
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Appendix  B:  Small  Scenario  Random  Seed  Run  Data 


The  20  variables  used  in  the  analysis  of  the  small  scenario  random  seed  runs  are 
presented  in  this  appendix.  Each  row  of  data  represents  the  output  from  one  run  of  AFM 
with  the  indicated  random  stream  and  the  original  small  scenario  file  63.  At  the  bottom  of 
each  column  of  data  are  the  mean,  standard  deviation,  and  coeflBcient  of  variation  for  the 
respective  variables.  The  coefficient  of  variation  is  a  statistical  tool  computed  by  dividing 
the  standard  deviation  by  the  mean.  In  this  case,  the  coefficients  of  variation  are  expressed 
as  a  percentage  of  the  respective  mean  values.  The  relative  magnitude  of  variation  caused 
by  changing  the  AFM  initial  random  seed  value,  across  all  variables,  can  be  understood  by 
looking  at  the  coefficients  of  variation. 

The  five  factor  scores  listed  in  this  appendix  were  computed  using  the  S  AS  factor 
analysis  procedure,  PROC  FACTOR  with  a  varimax  rotation.  The  number  of  factors 
selected  by  the  SAS,  for  this  set  of  20  variables,  was  6.  SAS  uses  the  Kaiser  criterion  (the 
number  of  eigenvalues  greater  than  or  equal  to  1)  to  determine  the  number  of  factors  to 
retain.  This  default  criterion  was  overridden  with  a  manual  command  to  retain  only  5 
factors  because  the  6-factor  results  were  unsatisfactory.  Two  of  the  sk  factors  had  only  1 
significant  loading  each.  The  eigenvalues  for  these  two  factors  were  fairly  low  (less  than 
1 .23)  compared  to  the  other  4  factors.  Retaining  one  less  factor  provided  a  more 
interpretable  factor  loadings  matrix  with  more  significant  eigenvalues  on  all  5  factors. 
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Appendix  C:  Large  Scenario  Random  Seed  Run  Data 


The  3 1  variables  used  in  the  analysis  of  the  large  scenario  random  seed  runs  are 
presented  in  this  appendbc.  Each  row  of  data  represents  the  output  from  one  run  of  AFM 
with  the  indicated  random  stream  and  the  original  large  scenario  file  63.  At  the  bottom  of 
each  column  of  data  are  the  mean,  standard  deviation,  and  coefficient  of  variation  for  the 
respective  variables.  The  coefficient  of  variation  is  a  statistical  tool  computed  by  dividing 
the  standard  deviation  by  the  mean.  In  this  case,  the  coefficients  of  variation  are  expressed 
as  a  percentage  of  the  respective  mean  values.  The  relative  magnitude  of  variation  caused 
by  changing  the  AFM  initial  random  seed  value,  across  all  variables,  can  be  understood  by 
looking  at  the  coefficients  of  variation. 

The  ten  factor  scores  listed  in  this  appendbc  were  computed  using  the  SAS  factor 
analysis  procedure,  PROC  FACTOR,  with  a  varimax  rotation.  The  number  of  factors 
selected  by  the  SAS,  for  this  set  of  3 1  variables,  was  10.  SAS  uses  the  Kaiser  criterion 
(the  number  of  eigenvalues  greater  than  or  equal  to  1)  to  determine  the  number  of  factors 
to  retain.  This  default  criterion  was  satisfactory. 
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Random 

Stream 


Mean! 


Std  Devi 


Coef  Var 


Throughput 


Outsize 
Tons  Pax 


12743  168343 


13191  165669 


13080  167957 


12646  163958 


13030  165566 


12837  167083 


12720  168513 


13058  166858 


12901  168331 


12463  171712 


13207  164803 


12604  171992 


12576  167000 


13161  164872 


12798  170357 


12593  167363 


12606  166008 


12984  167902 


12693  171863 


13115  165774 


12918  166929 


12812  167922 


12553  166439 


12566  164743 


13220  1714011 


12578  169688 


12918  174773 


12763  167894 


13187  164553 


12473  166029 


13115  166726 


13112  176542 


12467  169722 


13327  168883 


13231  163733 


12573  164588 


12579  169230 


12904  169845 


12606  168530 


12564  167822 


12605  175246 


12409  169628 


12412  163843 


12390  170569 


12845  167508 


12568  178478 


12732  172946 


13027  167217 


12576  171482 


12480  168712 


12790  168471 


266.9  3289.5 


2.1%  2.0% 


119971 


708.3 


0.6% 
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Random 

Stream 


1 


3 


8 


13 


15 


17 


20 


21 


22 


24 


25 


26 


29 


31 


32 


33 


37 


39 


41 


45 


46 


49 


50 


54 


55 


56 


61 


62 


64 


66 


67 


70 


71 


72 


73 


75 


80 


82 


83 


84 


85 


89 


90 


96 


97 


98 


99 


Mean 


Std  Dev 


Coef  Var 


G-Cycle 

Time 


29.28 


29.79 


29.60 


29.18 


29.05 


29.70 


29.08 


28.93 


29.68 


29.71 


29.28 


29.11 


29.70 


29.56 


30.28 


29.43 


28.95 


29.06 


29.35 


28.98 


29.70 


29.69 


29.34 


30.37 


28.76 


28.86 


28.96 


29.92 


29.37 


29.07 


29.05 


29.69 


29.51 


29.99 


29.21 


29.15 


29.33 


29.30 


29.24 


28.97 


29.36 


30.01 


30.22 


29.40 


29.36 


29.51 


30.00 


29.07 


29.04 


29.79 


KC-10 


Avg 

Payload 


33.15 


0.09221  9.41 


0.08771  9.3 


]EES^IKE]I 


0.09221  9.5 


0.09121  9.4 


0.09311  9.6 


0.08971  9.5 


EE 

lEESSSIKEll 


0.09251  9.5 


0.09151  9.4 


0.09131  9.4 


0.09441  9.4 


0.0920  9.5 


0.39 

0.91 

1.3% 

2.7% 

MTM/D 


0.1253 


0.1235 


0.1250 


0.1280 


0.1258 


0.1236 


0.1245 


0.1248 


0.1255 


0.1242 


0.1253 


0.1241 


0.1245 


0.1251 


0.1227 


0.1270 


0.1256 


0.1231 


0.1251 


0.1279 


0.1239 


0.1216 


0.1230 


0.1244 


0.1217 


0.1248 


0.1248 


0.1232 


0.1254 


0.1248 


0.1224 


0.1249 


0.1270 


0.1272 


0.1243 


0.1232 


0.1232 


0.1250 


0.1239 


0,1245 


0.1240 


0.1233 


0.1235 


0.1224 


0.1272 


0.1246 


0.1258 


0.1236 


0.1219 


0.1256 


36.46 

65.09 

0.1245 

0.45 

0.45 

0.0015 

1.2% 

0,7% 

1.2% 
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Appendix  D:  Fortran  Code  for  Generating  Perturbed  Files  63 


program  screendesign 


C23456789012345678901234567890123456789012345678901234567890123456789012 
This  program  uses  a  2"^4-l  factorial  design  to  test  the  relative 
importance  and  impact  of  the  four  TPFDD  (file63)  manipulations. 

The  design  encompasses  8  runs,  where  replications  for  each  run 
can  be  specified  by  the  user.  The  first  two  changes  to  the  file63 
are  random  effects,  but  will  be  treated  as  deterministic  effects 


c 

c 

for  this  analysis.  All  effects  are 
in  the  following  design: 

modelled  as  off(- 

-1)  or  on  (1) 

c 

c 

Design  point 

APOEAPOD 

ALDRDD 

10%  ton  inc 

out/ over  prop 

c 

0 

-1 

-1 

-1 

-1 

c 

1 

-1 

-1 

1 

1 

c 

2 

-1 

1 

-1 

1 

c 

3 

-1 

1 

1 

-1 

c 

4 

1 

-1 

-1 

1 

c 

5 

1 

-1 

1 

-1 

c 

6 

1 

1 

-1 

-1 

c 

7 

1 

1 

1 

1 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 


The  modification  to  the  file63  are  outlined  below: 

1)  APOEAPOD  pairs,  for  each  line,  are  swapped  with  a  randomly 
selected  pair  from  the  same  file63  that  has  a  distance 
within  +/-  X%  of  the  original  distance-  X  for  this 
screening  design  is  determined  by  the  user.  This 
modification  will  change  the  distribution  of  the  cargo  but 
should  leave  the  average  MTM/D  requirement  about  the  same. 
This  tests  the  model’s  sensitivity  to  varying  the  cargo 
distribution  between  network  resources. 

2)  ALD  and  RDD  were  adjusted  and  then  the  resultant  file63 
were  resorted  by  RDD  then  by  ALD  then  by  a  sequence  number 
that  reflects  where  the  line  was  in  the  original  sequence. 
This  sorting  keeps  all  the  lines  with  the  same  new  ALDRDD  in 
the  same  order  as  they  appeared  in  the  original  file63. 

This  tests  the  model’s  sensitivity  to  priorities  given  to 
each  line. 

3)  Increase  all  cargo  categories  and  pax  by  X%-  This  reflects 
a  slightly  larger  increase  in  rate  of  deliveries  required 
and  attempts  to  strain  the  system.  X  is  user  input. 

4)  Double  the  amount  of  outsize  cargo  and  subtract  an  equal 
amount  of  tons  from  the  oversize  cargo  (may  spread  over 
several  lines  if  necessary)  This  change  in  proportion  of 
outsize  to  oversize  reflects  changes  in  prepositioning 
strategies 

Written  by:  Capt  Glenn  G.  Rousseau,  USAF 
AFIT  Student,  GOA-96M 
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c 

c 


Variable  declarations  follow; 


integer  pcount , loops , 1, str, stream (1:99) , seed, lines , dayz , 

&  counter, newald, newrdd, aid, rdd, delta 

parameter  (lines=2000) 

character  £63*80, newlf63*80,new2f63*80,new3f63*80, £25*80, 

&  temp*2,oneline*71,des*80,sortpre£ix*10, 

&  new4f 63*80, newSf 63*80, new6£63*80,new7 £63* 80 

character*8  pairs (1: lines) , oldpair,newpair 

real  dpct, rnd, out, over , bulk, pax, incout, incover, incbulk, incpax, 

&  carry, spct , pdis t ( 1 : lines ) , out2 , over2 , incout2 , incover2 

£63=*f63* 

£25= *£25* 

des=*  screendesign.dat  * 

write (*,*)  *  Enter  integer  seed  greater  than  1:* 

read  (*,*)  seed 
rnd=rand (seed) 

open  (17,  £ile=des,  status=*new* ) 
str=0 

21  if  (str.eq.99)  then 

go  to  20 
endif 
str=str+l 
stream(str) =0 
go  to  21 

20  call  getpairs (pairs, pdist,pcount, lines, f25, f63) 

write (*,*)  *How  many  replications  of  the  design?* 
read  (*,*)  loops 

write (*,*)  'Allow  distance  to  vary  by  how  much  (value<l . 0) ? * 
read  (*,*)  dpct 

write {*,*)  'Maximum  number  of  days  around  current  ALD/RDD?* 
read  (*,*)  dayz 

write (*,*)  'Strain  increase  percentage  (value<l . 0) ? * 
read  (*,*)  spct 

do  10  i=l, loops 
counter=0 
carry=0. 0 
c 

c  Get  a  random  stream  value  (MASS  will  use  this  later  during  execution) 
c 

22  rnd=rand(0) 
str=int ( rnd*  99 . 0+1.0) 

if (stream(str) .eq.l)  then 
go  to  22 
end  if 


c  Save  unique  stream  value  in  screendesign.dat 
c 

stream(str) =1 

write(17, *  (i2.2)  *)  str 
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write (temp, ' (i2. 2) ' )  str 
c 

c  CiTBSit©  fil©n3.ni©s  with  coci©ci  ©xtsnsionj  d  iiiciic3.t©s  scjr©©ninQ’  dosiQ^ri/ 
c  1-7  in  the  next  character  indicates  which  run  from  the  screening 
c  design  this  file  contains,  and  the  last  two  digits  (temp)  store 
c  the  random  stream  to  use  when  executing  MASS 
c 

newlf 63=  *  f 63 . dl * // temp 
new2f 63=  *  f 63 . d2 ’ // temp 
new3f 63= ’ f 63 . d3 ’ // temp 
new4f 63=  *  f 63 . d4  * // temp 
newSf 63=  *  f 63 . d5  * // temp 
new6f 63= * f 63 . d6 * //temp 
new7f 63=  *  f 63 . d7  * // temp 
c 

c  Open  original  file  63 
c 

open (63, file=f 63, status=  *  old* ) 
c 

c  Create  7  perturbed  files  63  in  accordance  with  experimental  design 
c 

open ( 64 , f ile=newlf 63 , s tatus=  *  new  * ) 
open ( 65, file=new2f 63, status=  *  new* ) 
open (66, file=new3f 63, status= *  new* ) 
open ( 67 , f ile=new4  f 63 , s tatus=  *  new  * ) 
open ( 68 , f ile=new5f 63 , status=  *  new  * ) 
open ( 69 , f ile=new6f 63 , s tatus=  *  new  * ) 
open (70, file=new7f 63, status=*new* ) 

c 

c  Read  line  from  original  file  63 
c 

11  read  (63, *  (a71)  *,end=12)  oneline 
c 

c  Get  pertinent  variables  from  the  oneline  read 
c 

read  (oneline, 9999)  aid, rdd, oldpair, out, over, bulk, pax 

9999  format (t23, 2i3, a8, t44, 4f 7 • 1) 
c 

c  Randomly  select  new  APOEAPOD  pair 
c 

newpair=oldpair 

call  changeicaos (newpair,pairs,pdist,pcount, lines, dpct) 
c 

c  Randomly  select  perturbed  ALD/RDD  dates 
c 

rnd=rand ( 0 ) 

delta=int (rnd* (2 . 0*dayz+l . 0) ) -dayz 
newald=ald+delta 
if  (newald-lt- 0)  then 
newald=0 
end  if 

newrdd=rdd+delta 
if  (newrdd.lt . 0)  then 
newrdd=0 
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end  if 

if  (newrdd. gt . 999)  then 
newrdd=999 
end  if 

write (sortprefix, * (i3 . 3^ i3 . 3, i4 . 4 ) ’ )  rdd, aid, counter 
c 

c  Compute  double  the  outsize  value  and  subtract  from  oversize  colxiinn, 
c  carrying  balances  forward,  as  required, 
c 

carry=carry+out 

out2=out*2 

over2=over 

if  (carry. le.over2)  then 
over2=over2“carry 
carry=0 . 0 
else 

carry=carry-'Over2 
over2=0 . 0 
end  if 
c 

c  Compute  strained  increase  for  all  cargo  types 
c 

incout=out* (1+spct) 
incover=over* (1+spct) 
incbulk=bulk* (1+spct) 
incpax=float (int (pax* (1+spct) ) ) 
incout2=out2* (1+spct) 
incover2=over2* (1+spct) 

write (64,9997)  oneline (1:22), aid, rdd, oldpair , oneline (37:43), 

&  incout2 , incover2 , incbulk, incpax 

write  (65,9998)  sortpref ix, oneline (1:22), newald, newrdd, oldpair, 
&  oneline (37:43), out2 , over2 , bulk , pax 

write ( 66, 9998 )  sortpref ix, oneline (1:22), newald, newrdd, oldpair, 
&  oneline (37: 43) , incout, incover, incbulk, incpax 

write (67,9997)  oneline (1:22), aid, rdd, newpair , oneline (37:43), 

&  out2 , over2 , bulk, pax 

write  (68,9997)  oneline (1:22), aid, rdd, newpair, oneline (37:43), 

&  incout , incover, incbulk, incpax 

write (69,9998)  sortpref ix, oneline (1:22), newald, newrdd, newpair, 
&  oneline (37: 43) , out, over, bulk, pax 

write  (70,9998)  sortpref ix, oneline (1:22), newald, newrdd, newpair, 
&  oneline (37 : 43) , incout2, incover2, incbulk, incpax 

9997  format (a22, 2i3, a8, a7 , 4f7 . 1) 

9998  format (alO, a22 , 2i3, a8, a7, 4f7 . 1) 

go  to  11 
12  close  (63) 
close  (64) 
close  (65) 
close  (66) 
close  (67) 
close  (68) 
close  (69) 
close  (70) 

call  sortfile (new2f 63) 
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call  sortfile (new3f 63) 
call  sortfile (new6f63) 
call  sortfile (new7f 63) 

10  continue 
close (17) 

1100  format (a43, 4f7 . 1) 

end 

c 

c 

c 

c 

c 

subroutine  sortfile ( filename ) 

character  filename* 8 0 , s ortpre fix* 1 0 , oneline* 7 1 

IB  =  SYSTEM ('sort  '//filename//'  >  junk') 
open  (64,  file=*junk',  status=* old' ) 
open  (63,  file=filename,  status=' old' ) 

33  read  ( 64, ' (alO, a71) ' , end=34 )  sortprefix, oneline 
write  (63, ' (a71) ' )  oneline 

go  to  33 

34  close (63) 
close (64) 

IB  =  SYSTEM Crm  junk') 
return 
end 
c 
c 
c 
c 
c 

subroutine  getpairs (pairs, pdist,pcount,  lines, f 25, f63) 
integer  lines, pcount, i 
character*8  pairs (1: lines) ,  pair 
real  pdist (1: lines) 

character  f63*80,  f25*80,  oneline*71 
real  gcdist 
external  gcdist 

open  (63,  file=f63,  status='old' ) 
pcount=0 

110  read  (63,'(a71)*,  END=113)  oneline 
c  obtain  apoe/apod  icao  pair 

pair=oneline (29:36) 

c  if  pair  is  not  stored,  store  it  in  pairs  array 
i=l 

111  if  (i.le. pcount)  then 
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if  {pairs (i) . eq. pair)  then 
go  to  110 
end  if 
i=i+l 
go  to  111 
end  if 

pcount=pcount+l 
pairs (pcount) =pair 
pdist (pcount) =gcdist (pair, f25) 
go  to  110 

113  close  (63) 
return 
end 
c 
c 
c 
c 
c 

real  function  gcdist (pair, f25) 
integer  mog,  reg 
character  pair*8,  f25*80 

real  latl,  lonl,  lat2,  lon2,  pi,  d2r,  argl,  arg2,  arg3 

call  rdfile25 (pair (1:4), latl , lonl, mog, reg, f25 ) 
call  rdfile25 (pair (5 : 8) , lat2, lon2,mog, reg, f25) 

pi=acos (-1.0) 
d2r=pi/180.0 

argl  =  sin (latl*d2r) *sin (Iat2*d2r) 

arg2  =  cos (latl*d2r) *cos (Iat2*d2r) *cos ( {lon2-lonl) *d2r) 

arg3=argl+arg2 
if  (abs (arg3) .gt.1.0)  then 
arg3=l. 0 
end  if 

gcdist  =  60 • 0*acos (arg3) /d2r 

return 

end 

c 

c 

c 

c 

c 

subroutine  rdf ile25 (icao, lat , Ion, mog, reg, f25 ) 
integer  mog, deg, reg 
real  lat.  Ion,  min 

character*4  icao,  f25*80,  oneline*54 

open  (25,  file=f25,  status=* old* ) 

200  read  (25,*(a54)*,  END=205)  oneline 
if  (icao. eq. oneline (1: 4) )  then 

read{oneline(7:8) , * (i2)  * )  reg 
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read(oneline(9:15) , ' (i3,f4.1) ')  deg,  min 
lat=real  (deg)  +inin/ 60 . 0 
if  {oneline(16:16) .eq. 'S' )  then 
lat=  -1.0*lat 
end  if 

read{oneline(17:23), * (i3,f4.1) *)  deg,  min 
lon=real (deg) +min/ 60 . 0 
if  (oneline(24:24) -eq. 'EM  then 
lon=  -1.0*lon 
end  if 

read (oneline ( 41 : 45 ) , * (i5 ) M  mog 
if  (mog-gt.lOO)  then 

write(*,*)  'Subroutine  rdfile25  message:' 
write (*,*)  'Internally  reducing  large  MOG  for  ',icao, 
&  *  to  100  - ' 

mog=100 
end  if 

close  (25) 
return 

else 

go  to  200 
end  if 

205  close  (25) 

write(*,*)  'Subroutine  rdfile25  message:* 

write(*,*)  'ICAO  *,icao,*  not  found  in  file  25.* 

write (*,*)  'Execution  terminated.* 

stop 

end 

c 

c 

c 

c 

c 

subroutine  changeicaos (pair , pairs, pdist,pcount, lines, pet) 
integer  lines , pcount , i , choice , ccount ,  choices (1:200) 
character*8  pair, pairs (1: lines ) 

real  pdist (1: lines ) ,  pet,  maxdist,  mindist,  rnd 

do  300  i=l, pcount 
if  (pair. eq. pairs (i) )  then 
mindist=pdist  (i)  *  (1. 0“-pct) 
maxdist=pdist (i) * (1. 0+pct) 
go  to  301 
end  if 

300  continue 
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301  ccount=0 

do  302  i=l,pcount 

if  ( (inindist.le.pdist(i)  )  .and.  (maxdist.ge.pdist(i) )  )  then 
ccount=ccount+l 
choices (ccount)=i 
end  if 

302  continue 

303  rnd=rand(0) 

choice=int (rnd*ccount+l . 0) 

if  ( (choice. eq. 0) .or. (choice . gt . ccount) )  then 
go  to  303 
end  if 

pair-pairs (choice) 

return 

end 
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Appendix  E:  Small  Scenario  Perturbation  Run  Data 


The  20  variables  used  in  the  analysis  of  the  small  scenario  perturbation  runs  are 
presented  in  this  appendix.  Each  row  of  data  represents  the  output  from  one  run  of  AFM 
with  the  indicated  random  stream  and  a  perturbed  file  63  corresponding  to  the  design 
point  characteristics.  At  the  bottom  of  each  column  of  data  are  the  mean,  standard 
deviation,  and  coefficient  of  variation  for  the  respective  variables.  The  relative  magnitude 
of  variation  caused  by  perturbing  the  file63,  can  be  understood  by  looking  at  the 
coefficients  of  variation.  As  compared  to  relatively  low  coefficients  of  variation  from 
random  seed  effects  (See  Appendix  B),  most  of  the  variables  seemed  to  be  significantly 
affected  by  the  file63  perturbations.  Note  that  total  cargo  tons  throughput  and  percent  of 
shipments  delivered  ontime  were  not  affected  much  by  the  perturbation  schemes 
(coefficients  of  variation  <  5.0%),  indicating  their  robustness  across  many  different 
TPFDD  profiles. 

The  four  factor  scores  listed  in  this  appendbc  were  computed  using  the  SAS  factor 
analysis  procedure,  PROC  FACTOR,  with  a  varimax  rotation.  The  number  of  factors 
selected  by  the  SAS,  using  Kaiser’s  criterion,  was  4,  which  was  satisfactory.  Loadings 
matrix  interpretation  and  plots  of  these  factors  scores  are  provided  in  Chapter  IV. 
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Appendix  F:  Large  Scenario  Perturbation  Run  Data 


The  29  variables  used  in  the  analysis  of  the  large  scenario  perturbation  runs  are 
presented  in  this  appendix.  Though  the  3 1  variables  used  in  the  random  seed  runs  were 
desired  for  this  section  also,  C-17  ground  cycle  times  did  not  consistently  output  from 
AFM  and  had  to  be  discarded.  Also,  C-141B  MTM/D  per  aircraft  was  uniquely 
independent  of  all  other  variables  and  was  removed. 

Each  row  of  data  in  this  appendix  represents  the  output  from  one  run  of  AFM  with 
the  indicated  random  stream  and  a  perturbed  file  63  corresponding  to  the  design  point 
characteristics.  At  the  bottom  of  each  column  of  data  are  the  mean,  standard  deviation, 
and  coefficient  of  variation  for  the  respective  variables.  As  compared  to  relatively  low 
coefficients  of  variation  from  random  seed  effects  (See  Appendix  C),  most  of  the  variables 
seemed  to  be  unaffected  by  the  file63  perturbations.  Note  that  total  cargo  tons 
throughput  was  again  relatively  unaffected  by  the  perturbation  schemes,  as  in  the  small 
scenario  (See  Appendbc  E).  Also  note,  however,  that  percent  of  shipments  delivered 
ontime  was  significantly  affected  by  the  perturbation  schemes  in  the  large  scenario,  but  not 
in  the  small  scenario. 

The  eight  factor  scores  listed  in  this  appendk  were  computed  using  the  S  AS  factor 
analysis  procedure,  PROC  FACTOR,  with  a  varimax  rotation.  The  number  of  factors 
selected  by  the  SAS,  using  Kaiser’s  criterion,  was  8,  which  was  satisfactory.  Loadings 
matrix  interpretation  and  plots  of  these  factors  scores  are  provided  in  Chapter  IV. 
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